Vision testing via prediction-based setting of initial stimuli characteristics for user interface locations

ABSTRACT

In some embodiments, initial feedback indicating threshold characteristics (under which a user sees initial stimuli presented on a user interface) may be provided to a prediction model, and a set of predicted characteristics (for a set of locations of the user interface) and a set of confidence scores associated with the set of locations may be obtained via the prediction model. Based on the set of confidence scores, one or more locations may be selected to be tested during a visual test presentation. As an example, the locations may be selected over one or more other locations of the set of locations based on the set of confidence scores. Based on predicted characteristics associated with the selected locations, stimuli may be presented at the selected locations during the visual test presentation. Visual defect information for the user may be generated based on feedback from the visual test presentation.

CROSS REFERENCE TO RELATED APPLICATIONS

This application is a continuation of U.S. patent application Ser. No.17/083,043, filed on Oct. 28, 2020, the contents of which are herebyincorporated by reference in its entirety.

FIELD OF THE INVENTION

The invention relates to facilitating vision testing and vision defectdetermination therefrom.

BACKGROUND

Current vision defect determination requires the testing of manylocations of a user's field of view. Oftentimes, each location may betested over a series of characteristics, making the testing process slowand inefficient. For example, a vision test may include testing up toseveral thousand combinations (e.g., over 70 locations of the user'sfield of view under a series of 50 different characteristics or othercombinations). These and other drawbacks exist.

SUMMARY

Aspects of the invention relate to methods, apparatuses, and/or systemsfor facilitating vision testing via confidence-based selection of atesting location subset.

In some embodiments, initial feedback indicating thresholdcharacteristics (under which a user sees initial stimuli presented on auser interface) may be provided to a prediction model, and a set ofpredicted characteristics (for a set of locations of the user interface)and a set of confidence scores associated with the set of locations maybe obtained via the prediction model. Based on the set of confidencescores, one or more locations may be selected to be tested during avisual test presentation. As an example, the locations may be selectedover one or more other locations of the set of locations based on theset of confidence scores. Based on predicted characteristics associatedwith the selected locations, stimuli may be presented at the selectedlocations during the visual test presentation. Visual defect informationfor the user may be generated based on feedback from the visual testpresentation.

Various other aspects, features, and advantages of the invention will beapparent through the detailed description of the invention and thedrawings attached hereto. It is also to be understood that both theforegoing general description and the following detailed description areexemplary and not restrictive of the scope of the invention. As used inthe specification and in the claims, the singular forms of “a,” “an,”and “the” include plural referents unless the context clearly dictatesotherwise. In addition, as used in the specification and the claims, theterm “or” means “and/or” unless the context clearly dictates otherwise.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1A illustrates a system for facilitating vision testing, visiondefect determination, or modification related to a vision of a user, inaccordance with one or more embodiments.

FIG. 1B illustrates a system implementing a machine learning model tofacilitate vision testing, vision defect determination, or modificationrelated to a vision of a user, in accordance with one or moreembodiments.

FIGS. 1C-1F illustrate views of example spectacles devices, inaccordance with one or more embodiments.

FIG. 2 illustrates an example vision system, in accordance with one ormore embodiments.

FIG. 3 illustrates a device with a vision correction frameworkimplemented on an image processing device and a wearable spectaclesdevice, in accordance with one or more embodiments.

FIG. 4 illustrates an example process including a testing mode and avisioning mode, in accordance with one or more embodiments.

FIG. 5 illustrates an example process including a testing mode and avisioning mode, in accordance with one or more embodiments.

FIGS. 6A-6C illustrate an example assessment protocol for a testing modeprocess including pupil tracking, in accordance with one or moreembodiments.

FIGS. 7A-7C illustrate an example assessment protocol for a testing modeprocess including pupil tracking, in accordance with one or moreembodiments.

FIG. 8 illustrates a workflow including a testing module that generatesand presents a plurality of visual stimuli to a user through a wearablespectacles device, in accordance with one or more embodiments.

FIG. 9 illustrates a testing mode process, in accordance with one ormore embodiments.

FIG. 10 illustrates a process for an artificial intelligence correctivealgorithm mode that may be implemented as part of the testing mode, inaccordance with one or more embodiments.

FIG. 11 illustrates a test image, in accordance with one or moreembodiments.

FIG. 12 illustrates development of a simulated vision image includingoverlaying an impaired visual field on a test image for presentation toa subject, in accordance with one or more embodiments.

FIG. 13 illustrates examples of different correction transformationsthat may be applied to an image and presented to a subject, inaccordance with one or more embodiments.

FIG. 14 illustrates example translation methods, in accordance with oneor more embodiments.

FIG. 15 illustrates an example of a machine learning framework, inaccordance with one or more embodiments.

FIG. 16 illustrates a process of an AI system of a machine learningframework, in accordance with one or more embodiments.

FIG. 17 illustrates an example transformation of a test image, inaccordance with one or more embodiments.

FIG. 18 illustrates an example translation of a test image, inaccordance with one or more embodiments.

FIG. 19 is a graphical user interface illustrating various aspects of animplementation of an AI system, in accordance with one or moreembodiments.

FIG. 20 illustrates a framework for an AI system including afeed-forward neural network, in accordance with one or more embodiments.

FIGS. 21-22 illustrate example testing mode processes of an AI systemincluding a neural network and an AI algorithm optimization process,respectively, in accordance with one or more embodiments.

FIG. 23 illustrates an example process implementing testing andvisioning modes, in accordance with one or more embodiments.

FIG. 24A illustrates a wearable spectacles device comprising customreality wearable spectacles that allow an image from the environment topass through a transparent portion of the wearable spectacles' display,where the transparent portion corresponds to a peripheral region of theuser's visual field, and where other portions of the wearablespectacles' display are opaque portions, in accordance with one or moreembodiments.

FIG. 24B illustrates a wearable spectacles device comprising customreality wearable spectacles that allow an image from the environment topass through a transparent portion of the wearable spectacles' display,where the transparent portion corresponds to a central region of theuser's visual field, and where other portions of the wearablespectacles' display are opaque portions, in accordance with one or moreembodiments.

FIG. 24C illustrates an alignment between visual field plane, a remappedimage plane, and a selective transparency screen plane using eyetracking, in accordance with one or more embodiments.

FIG. 25A illustrates a use case of a visual test presentation beingdisplayed to a patient without crossed eyes, in accordance with one ormore embodiments.

FIG. 25B illustrates a use case of a visual test presentation beingdisplayed to a patient with crossed eyes, in accordance with one or moreembodiments.

FIG. 25C-25I illustrate automated measurement and correction of doublevision, in accordance with one or more embodiments.

FIG. 25J-25L illustrate binocular vision testing and results of suchtesting, in accordance with one or more embodiments.

FIG. 25M-25N illustrate stereopsis testing, in accordance with one ormore embodiments.

FIG. 26 illustrates a normal binocular vision for a subject where amonocular image from the left eye and from the right eye are combinedinto a single perceived image having a macular central area and aperipheral visual field area surrounding the central area;

FIG. 27 illustrates a tunnel vision condition wherein a peripheral areais not visible to a subject;

FIG. 28 illustrates an image shifting technique to enhance vision or tocorrect a tunnel vision condition, in accordance with one or moreembodiments.

FIG. 29 illustrates an image resizing transformation technique toenhance vision or preserve central visual acuity while expanding thevisual field, in accordance with one or more embodiments.

FIG. 30 illustrates a binocular view field expansion technique, inaccordance with one or more embodiments.

FIG. 31A illustrates a technique for assessing dry eye and cornealirregularities including projecting a pattern onto the corneal surfaceand imaging the corneal surface reflecting the pattern, in accordancewith one or more embodiments.

FIG. 31B schematically illustrates presentation of a reference imagecomprising a grid displayed to a subject or projected onto a cornea orretina of the subject via wearable spectacles, in accordance with one ormore embodiments.

FIG. 31C illustrates an example grid for manipulation by a subject, inaccordance with one or more embodiments.

FIG. 31D illustrates an example manipulation of the grid illustrated inFIG. 31C, in accordance with one or more embodiments.

FIG. 31E illustrates a scene as it should be perceived by the subject,in accordance with one or more embodiments.

FIG. 31F illustrates an example corrected visual field that whenprovided to a subject with a visual distortion determined by the gridtechnique results in that subject perceiving the visual field as shownFIG. 31E, in accordance with one or more embodiments.

FIG. 31G illustrates a display including a manipulatable grid onto whicha subject may communicate distortions within a visual field, inaccordance with one or more embodiments.

FIG. 32 is an image of a corneal surface reflecting a pattern projectedonto the corneal surface, in accordance with one or more embodiments.

FIG. 33 illustrates an example of a normal pattern reflection, inaccordance with one or more embodiments.

FIG. 34 illustrates an example of an abnormal pattern reflection, inaccordance with one or more embodiments.

FIGS. 35A-35E illustrates a visual test presentation using a dynamicfixation point, in accordance with one or more embodiments.

FIG. 35F illustrates a flowchart related to a process for facilitating avisual test presentation using a dynamic fixation point, in accordancewith one or more embodiments.

FIG. 35G-35I illustrate vision testing involving monitoring of whethereye movement remains within or occurs outside a bounding structure orregion when a stimulus is presented, in accordance with one or moreembodiments.

FIG. 35J illustrates a visual test presentation including multiplecontrast staircase stimuli and stimuli sequences at predeterminedlocations, in accordance with one or more embodiments.

FIG. 36 illustrates a timing diagram showing operations of a testingsequence at one stimulus location, in accordance with one or moreembodiments.

FIG. 37 illustrates calculation of widths and heights of pixels boundingthe largest bright field, in accordance with one or more embodiments.

FIG. 38 illustrate test images used to test four main quadrants of avisual field, in accordance with one or more embodiments.

FIG. 39A illustrates an example visual field view prior to remapping, inaccordance with one or more embodiments.

FIG. 39B illustrates an example visual field view following remapping,in accordance with one or more embodiments.

FIGS. 40A-40C illustrates an example custom reality spectacles device,in accordance with one or more embodiments.

FIG. 41 shows a flowchart of a method of facilitating modificationrelated to a vision of a user via a prediction model, in accordance withone or more embodiments.

FIG. 42 shows a flowchart of a method of facilitating an increase in afield of view of a user via combination of portions of multiple imagesof a scene, in accordance with one or more embodiments.

FIG. 43 shows a flowchart of a method of facilitating enhancement of afield of view of a user via one or more dynamic display portions on oneor more transparent displays, in accordance with one or moreembodiments.

FIGS. 44A-44B show a stimulus presented at a visual field location atdifferent times under two different characteristics, in accordance withone or more embodiments.

FIG. 44C shows a set of stimuli presented at a set of locations of auser interface, in accordance with one or more embodiments.

FIG. 45 shows examples of common patterns of visual defects reflectingdata used to train or configure one or more prediction models, inaccordance with one or more embodiments.

FIGS. 46A-46F shows portions of a vision test facilitated byprediction-based setting of initial stimuli characteristics, inaccordance with one or more embodiments.

FIGS. 47A-47F shows portions of a vision test facilitated byprediction-based setting of initial stimuli characteristics andconfidence-based selection of a testing location subset, in accordancewith one or more embodiments.

FIG. 48 shows a flowchart of a method of facilitating vision testing viapattern-based selection of a testing location, in accordance with one ormore embodiments.

FIG. 49 shows a flowchart of a method of facilitating vision testing viaconfidence-based selection of a testing location subset, in accordancewith one or more embodiments.

DETAILED DESCRIPTION

In the following description, for the purposes of explanation, numerousspecific details are set forth in order to provide a thoroughunderstanding of the embodiments of the invention. It will beappreciated, however, by those having skill in the art that theembodiments of the invention may be practiced without these specificdetails or with an equivalent arrangement. In other cases, well-knownstructures and devices are shown in block diagram form in order to avoidunnecessarily obscuring the embodiments of the invention.

FIG. 1A shows a system 100 for facilitating vision testing, visiondefect determination, or modification related to a vision of a user, inaccordance with one or more embodiments. As shown in FIG. 1A, system 100may include server(s) 102, client device 104 (or client devices 104a-104 n), or other components. Server 102 may include configurationsubsystem 112, model manager subsystem 114, or other components. Clientdevice 104 may include testing subsystem 122, visioning subsystem 124,or other components. Each client device 104 may include any type ofmobile terminal, fixed terminal, or other device. By way of example,client device 104 may include a desktop computer, a notebook computer, atablet computer, a smartphone, a wearable device, or other clientdevice. Users may, for instance, utilize one or more client devices 104to interact with one another, one or more servers, or other componentsof system 100. Additionally, or alternatively, system 100 may compriseone or more components described in U.S. patent application (AttorneyDocket No.: 054862-0515634), entitled “Systems and Methods for VisualField Testing in Head-Mounted Displays” and filed Oct. 28, 2020, thecontents of which are hereby incorporated by reference in its entirety.

It should be noted that, while one or more operations are describedherein as being performed by particular components of client device 104,those operations may, in some embodiments, be performed by othercomponents of client device 104 or other components of system 100. As anexample, while one or more operations are described herein as beingperformed by components of client device 104, those operations may, insome embodiments, be performed by components of server 102. It shouldalso be noted that, while one or more operations are described herein asbeing performed by particular components of server 102, those operationsmay, in some embodiments, be performed by other components of server 102or other components of system 100. As an example, while one or moreoperations are described herein as being performed by components ofserver 102, those operations may, in some embodiments, be performed bycomponents of client device 104. It should further be noted that,although some embodiments are described herein with respect to machinelearning models, other prediction models (e.g., statistical models orother analytics models) may be used in lieu of or in addition to machinelearning models in other embodiments (e.g., a statistical modelreplacing a machine learning model and a non-statistical model replacinga non-machine-learning model in one or more embodiments).

In some embodiments, system 100 may provide a visual test presentationto a user, where the presentation including a set of stimuli (e.g.,light stimuli, text, or images displayed to the user). During thepresentation (or after the presentation), system 100 may obtain feedbackrelated to the set of stimuli (e.g., feedback indicating whether or howthe user sees one or more stimuli of the set). As an example, thefeedback may include an indication of a response of the user to one ormore stimuli (of the set of stimuli) or an indication of a lack ofresponse of the user to such stimuli. The response (or lack thereof) mayrelate to an eye movement, a gaze direction, a pupil size change, or auser modification of one or more stimuli or other user input (e.g., theuser's reaction or other response to the stimuli). As another example,the feedback may include an eye image captured during the visual testpresentation. The eye image may be an image of a retina of the eye(e.g., the overall retina or a portion thereof), an image of a cornea ofthe eye (e.g., the overall cornea or a portion thereof), or other eyeimage.

In some embodiments, system 100 may determine one or more defectivevisual field portions of a visual field of a user (e.g., an automaticdetermination based on feedback related to a set of stimuli displayed tothe user or other feedback). As an example, a defective visual fieldportion may be one of the visual field portions of the user's visualfield that fails to satisfy one or more vision criteria (e.g., whetheror an extent to which the user senses one or more stimuli, an extent oflight sensitivity, distortion, or other aberration, or other criteria).In some embodiments, system 100 may provide an enhanced image or adjustone or more configurations of a wearable device based on thedetermination of the defective visual field portions. As an example, theenhanced image may be generated or displayed to the user such that oneor more given portions of the enhanced image (e.g., a region of theenhanced image that corresponds to a macular region of the visual fieldof an eye of the user or to a region within the macular region of theeye) are outside of the defective visual field portion. As anotherexample, a position, shape, or size of one or more display portions ofthe wearable device, a brightness, contrast, saturation, or sharpnesslevel of such display portions, a transparency of such display portions,or other configuration of the wearable device may be adjusted based onthe determined defective visual field portions.

In some embodiments, one or more prediction models may be used tofacilitate determination of vision defects (e.g., light sensitivities,distortions, or other aberrations), determination of modificationprofiles (e.g., correction/enhancement profiles that includemodification parameters or functions) to be used to correct or enhance auser's vision, generation of enhanced images (e.g., derived from liveimage data), or other operations. In some embodiments, the predictionmodels may include one or more neural networks or other machine learningmodels. As an example, neural networks may be based on a largecollection of neural units (or artificial neurons). Neural networks mayloosely mimic the manner in which a biological brain works (e.g., vialarge clusters of biological neurons connected by axons). Each neuralunit of a neural network may be connected with many other neural unitsof the neural network. Such connections can be enforcing or inhibitoryin their effect on the activation state of connected neural units. Insome embodiments, each individual neural unit may have a summationfunction which combines the values of all its inputs together. In someembodiments, each connection (or the neural unit itself) may have athreshold function such that the signal must surpass the thresholdbefore it propagates to other neural units. These neural network systemsmay be self-learning and trained, rather than explicitly programmed, andcan perform significantly better in certain areas of problem solving, ascompared to traditional computer programs. In some embodiments, neuralnetworks may include multiple layers (e.g., where a signal pathtraverses from front layers to back layers). In some embodiments, backpropagation techniques may be utilized by the neural networks, whereforward stimulation is used to reset weights on the “front” neuralunits. In some embodiments, stimulation and inhibition for neuralnetworks may be more free-flowing, with connections interacting in amore chaotic and complex fashion.

As an example, with respect to FIG. 1B, machine learning model 162 maytake inputs 164 and provide outputs 166. In one use case, outputs 166may be fed back to machine learning model 162 as input to train machinelearning model 162 (e.g., alone or in conjunction with user indicationsof the accuracy of outputs 166, labels associated with the inputs, orwith other reference feedback information). In another use case, machinelearning model 162 may update its configurations (e.g., weights, biases,or other parameters) based on its assessment of its prediction (e.g.,outputs 166) and reference feedback information (e.g., user indicationof accuracy, reference labels, or other information). In another usecase, where machine learning model 162 is a neural network, connectionweights may be adjusted to reconcile differences between the neuralnetwork's prediction and the reference feedback. In a further use case,one or more neurons (or nodes) of the neural network may require thattheir respective errors be sent backward through the neural network tothem to facilitate the update process (e.g., backpropagation of error).Updates to the connection weights may, for example, be reflective of themagnitude of error propagated backward after a forward pass has beencompleted. In this way, for example, the prediction model may be trainedto generate better predictions.

In some embodiments, upon obtaining feedback related to a set of stimuli(displayed to a user), feedback related to one or more eyes of the user,feedback related to an environment of the user, or other feedback,system 100 may provide the feedback to a prediction model, and theprediction model may be configured based on the feedback. As an example,the prediction model may be automatically configured for the user basedon (i) an indication of a response of the user to one or more stimuli(of the set of stimuli), (ii) an indication of a lack of response of theuser to such stimuli, (iii) an eye image captured during the visual testpresentation, or other feedback (e.g., the prediction model may bepersonalized toward the user based on the feedback from the visual testpresentation). As another example, the prediction model may be trainedbased on such feedback and other feedback from other users to improveaccuracy of results provided by the prediction model. In someembodiments, upon the prediction model being configured (e.g., for theuser), system 100 may provide live image data or other data to theprediction model to obtain an enhanced image (derived from the liveimage data) and cause the enhanced image to be displayed. As an example,a wearable device of system 100 may obtain a live video stream from oneor more cameras of the wearable device and cause the enhanced image tobe displayed on one or more displays of the wearable device. In someembodiments, the wearable device may obtain the enhanced image (e.g., afile or other data structure representing the enhanced image) from theprediction model. In some embodiments, the wearable device may obtain amodification profile (e.g., modification parameters or functions) fromthe prediction model, and generate the enhanced image based on the livevideo stream and the modification profile. In one use case, themodification profile may include modification parameters or functionsused to generate the enhanced image from the live image data (e.g.,parameters of functions used to transform or modify the live image datainto the enhanced image). Additionally, or alternatively, themodification profile may include modification parameters or functions todynamically configure one or more display portions (e.g., dynamicadjustment of transparent or opaque portions of a transparent display,dynamic adjustment of projecting portions of a projector, etc.).

In some embodiments, system 100 may facilitate enhancement of a field ofview of a user via one or more dynamic display portions (e.g.,transparent display portions on a transparent display, projectingportions of a projector, etc.). As an example, with respect to atransparent display, the dynamic display portions may include one ormore transparent display portions and one or more other display portions(e.g., of a wearable device or other device). System 100 may cause oneor more images to be displayed on the other display portions. As anexample, a user may see through the transparent display portions of atransparent display, but may not be able to see through the otherdisplay portions and instead sees the image presentation on the otherdisplay portions (e.g., around or proximate the transparent displayportions) of the transparent display. In one use case, live image datamay be obtained via the wearable device, and an enhanced image may begenerated based on the live image data and displayed on the otherdisplay portions of the wearable device. In some embodiments, system 100may monitor one or more changes related to one or more eyes of the userand cause, based on the monitoring, an adjustment of the transparentdisplay portions of the transparent display. As an example, themonitored changes may include an eye movement, a change in gazedirection, a pupil size change, or other changes. One or more positions,shapes, sizes, transparencies, or other aspects of the transparentdisplay portions of the wearable device may be automatically adjustedbased on the monitored changes. In this way, for example, system 100 mayimprove mobility without restriction (or at least reducing restrictions)on eye movements, gaze direction, pupil responses, or other changesrelated to the eye.

In some embodiments, system 100 may facilitate an increase in a field ofview of a user via combination of portions of multiple images of a scene(e.g., based on feedback related to a set of stimuli displayed to theuser or other feedback), system 100 may obtain a plurality of images ofa scene. System 100 may determine a region common to the images, and,for each image of the images, determine a region of the image divergentfrom a corresponding region of at least another image of the images. Insome embodiments, system 100 may generate or display an enhanced imageto a user based on the common region and the divergent regions. As anexample, the common region and the divergent regions may be combined togenerate the enhanced image to include a representation of the commonregion and representations of the divergent regions. The common regionmay correspond to respective portions of the images that have the sameor similar characteristics as one another, and each divergent region maycorrespond to a portion of one of the images that is distinct from allthe other corresponding portions of the other images. In one scenario, adistinct portion of one image may include a part of the scene that isnot represented in the other images. In this way, for example, thecombination of the common region and the divergent region into anenhanced image increase the field of view otherwise provided by each ofthe images, and the enhanced image may be used to augment the user'svision.

In some embodiments, system 100 may generate a prediction indicatingthat an object will come in physical contact with a user and cause analert to be displayed based on the physical contact prediction (e.g., analert related to the object is displayed on a wearable device of theuser). In some embodiments, system 100 may detect an object in adefective visual field portion of a visual field of a user and cause thealert to be displayed based on (i) the object being in the defectivevisual field portion, (ii) the physical contact prediction, or (iii)other information. In some embodiments, system 100 may determine whetherthe object is outside (or not sufficiently in) any image portion of anenhanced image (displayed to the user) that corresponds to at least onevisual field portions satisfying one or more vision criteria. In one usecase, no alert may be displayed (or a lesser-priority alert may bedisplayed) when the object is determined to be within (or sufficientlyin) an image portion of the enhanced image that corresponds to theuser's intact visual field portion (e.g., even if the object ispredicted to come in physical contact with the user). On the other hand,if the object in the defective visual field portion is predicted to comein physical contact with the user, and it is determined that the objectis outside (or not sufficiently in) the user's intact visual fieldportion, an alert may be displayed on the user's wearable device. Inthis way, for example, the user can rely on the user's own intact visualfield to avoid incoming objects within the user's intact visual field,thereby mitigating the risk of dependence on the wearable device (e.g.,through habit forming) for avoidance of such incoming objects. It shouldbe noted, however, that, in other use cases, an alert related to theobject may be displayed based on the physical contact predictionregardless of whether the object is within the user's intact visualfield.

In some embodiments, with respect to FIG. 1C, client device 104 mayinclude a spectacles device 170 forming a wearable device for a subject.In some embodiments, the spectacles device 170 may be a part of avisioning system as described herein. The spectacles device 170 includesa left eyepiece 172 and a right eyepiece 174. Each eyepiece 172 and 174may contain and/or associate with a digital monitor configured todisplay (e.g., provide on a screen or project onto an eye) recreatedimages to a respective eye of the subject. In various embodiments,digital monitors may include a display screen, projectors, and/orhardware to generate the image display on the display screen or projectimages onto an eye (e.g., a retina of the eye). It will be appreciatedthat digital monitors comprising projectors may be positioned at otherlocations to project images onto an eye of the subject or onto aneyepiece comprising a screen, glass, or other surface onto which imagesmay be projected. In one embodiment, the left eyepiece 172 and righteyepiece 174 may be positioned with respect to the housing 176 to fit anorbital area on the subject such that each eyepiece 172, 174 is able tocollect data and display/project image data, which in a further exampleincludes displaying/projecting image data to a different eye.

Each eyepiece 172, 174 may further includes one or more inward directedsensors 178, 180, which may be inward directed image sensors. In anexample, inward directed sensors 178, 180 may include infrared cameras,photodetectors, or other infrared sensors, configured to track pupilmovement and to determine and track visual axes of the subject. Theinward directed sensors 178, 180 (e.g., comprising infrared cameras) maybe located in lower portions relative to the eyepieces 172, 174, so asto not block the visual field of the subject, neither their real visualfield nor a visual field displayed or projected to the subject. Theinward directed sensors 178, 180 may be directionally aligned to pointtoward a presumed pupil region for better pupil and/or line of sighttracking. In some examples, the inward directed sensors 178, 180 may beembedded within the eyepieces 172, 174 to provide a continuous interiorsurface.

FIG. 1D illustrates a front view of the spectacles device 170, showingthe front view of the eyepieces 172, 174, where respective outwarddirected image sensors 182, 184 comprising field of vision cameras arepositioned. In other embodiments, fewer or additional outward directedimage sensors 182, 184 may be provided. The outward directed imagesensors 182, 184 may be configured to capture continuous images. Thespectacles device 170 or associated vision system may be furtherconfigured to then correct and/or enhance the images, which may be in acustomized manner based on the optical pathologies of the subject. Thespectacles device 170 may further be configured to display the correctedand/or enhanced image to the subject via the monitors in a visioningmode. For example, the spectacles device may generate the correctedand/or enhanced image on a display screen associated with the eyepieceor adjacent region, project the image onto a display screen associatedwith the eyepiece or adjacent region, or project the image onto one ormore eyes of the subject.

FIGS. 1E-1F illustrate other examples of spectacles device 170. Withrespect to FIGS. 1E-1F, spectacles device 170 includes a high-resolutioncamera (or cameras) 192, a power unit 193, a processing unit 194, aglass screen 195, a see-through display 196 (e.g., a transparentdisplay), an eye tracking system 197, and other components.

In some embodiments, the spectacles device 170 may include a testingmode. In an example testing mode, the inward directed sensors 178, 180track pupil movement and perform visual axis tracking (e.g., line ofsight) in response to a testing protocol. In this or another example,the inward directed sensors 178, 180 may be configured to capture areflection of a pattern reflected on the cornea and/or retina to detectdistortions and irregularities of the cornea or the ocular opticalsystem.

Testing mode may be used to perform a visual assessments to identifyocular pathologies, such as, high and/or low order aberrations,pathologies of the optic nerve such as glaucoma, optic neuritis, andoptic neuropathies, pathologies of the retina such as maculardegeneration, retinitis pigmentosa, pathologies of the visual pathway asmicrovascular strokes and tumors and other conditions such aspresbyopia, strabismus, high and low optical aberrations, monocularvision, anisometropia and aniseikonia, light sensitivity, anisocorianrefractive errors, and astigmatism. In the testing mode, data may becollected for the particular subject and used to correct captured imagesbefore those images are displayed, which may include projected asdescribed herein, to the subject by the monitors.

In some examples, external sensors may be used to provide further datafor assessing visual field of the subject. For example, data used tocorrect the captured image may be obtained from external testingdevices, such as visual field testing devices, aberrometers,electro-oculograms, or visual evoked potential devices. Data obtainedfrom those devices may be combined with pupil or line of sight trackingfor visual axis determinations to create one or more modificationprofiles used to modify the images being projected or displayed to auser (e.g., correction profiles, enhancement profiles, etc., used tocorrect or enhance such images).

The spectacles device 170 may include a visioning mode, which may be inaddition to or instead of a testing mode. In visioning mode, one or moreoutward directed image sensors 182, 184 capture images that aretransmitted to an imaging processor for real-time image processing. Theimage processor may be embedded with the spectacles device 170 or may beexternal thereto, such as associated with an external image processingdevice. The imaging processor may be a component of a visioning moduleand/or include a scene processing module as described elsewhere herein.

The spectacles device 170 may be communicatively coupled with one ormore imaging processor through wired or wireless communications, such asthrough a wireless transceiver embedded within the spectacles device170. An external imaging processor may include a computer such as alaptop computer, tablet, mobile phone, network server, or other computerprocessing devices, centralized or distributed, and may be characterizedby one or more processors and one or more memories. In the discussedexample, the captured images are processed in this external imageprocessing device; however, in other examples, the captured images maybe processed by an imaging processor embedded within the digitalspectacles. The processed images (e.g., enhanced to improve functionalvisual field or other vision aspects and/or enhanced to correct for thevisual field pathologies of the subject) are then transmitted to thespectacles device 170 and displayed by the monitors for viewing by thesubject.

In an example operation of a vision system including the spectaclesdevice, real-time image processing of captured images may be executed byan imaging processor (e.g., using a custom-built MATLAB (MathWorks,Natick, Mass.) code) that runs on a miniature computer embedded in thespectacles device. In other examples, the code may be run on an externalimage processing device or other computer wirelessly networked tocommunicate with the spectacles device. In one embodiment, the visionsystem, including the spectacles device, image processor, and associatedinstructions for executing visioning and/or testing modes, which may beembodied on the spectacles device alone or in combination with one ormore external devices (e.g., laptop computer) may be operated in twomodes, a visioning mode and a separate testing mode.

In some embodiments, with respect to FIG. 2, system 100 may includevision system 200, which includes a spectacles device 202communicatively coupled to a network 204 for communicating with a server206, mobile cellular phone 208, or personal computer 210, any of whichmay contain a visional correction framework 212 for implementing theprocessing techniques herein, such as image processing techniques, whichmay include those with respect to the testing mode and/or visioningmode. In the illustrated example, the visional correction framework 212includes a processor and a memory storing an operating system andapplications for implementing the techniques herein, along with atransceiver for communicating with the spectacles device 202 over thenetwork 204. The framework 212 contains a testing module 214, whichincludes a machine learning framework in the present example. Themachine learning framework may be used along with a testing protocolexecuted by the testing module, to adaptively adjust the testing mode tomore accurately assess ocular pathologies, in either a supervised orunsupervised manner. The result of the testing module operation mayinclude development of a customized vision correction model 216 for asubject 218.

A visioning module 220, which in some embodiments may also include amachine learning framework having accessed customized vision correctionmodels, to generate corrected visual images for display by thespectacles device 202. The vision correction framework 212 may alsoinclude a scene processing module which may process images for useduring testing mode and/or visioning mode operations and may includeoperations described above and elsewhere herein with respect to aprocessing module. As described above and elsewhere herein, in someembodiments, the spectacles device 202 may include all or a portion ofthe vision correction framework 212.

In the testing mode, the spectacles device 170 or 202, and in particularthe inward directed image sensors comprising tracking cameras, which maybe positioned along an interior of the spectacles device 170 or 202, maybe used to capture pupil and visual axis tracking data that is used toaccurately register the processed images on the subject's pupil andvisual axis.

In some embodiments, with respect to FIG. 3, system 100 may include avision system 300, which includes a vision correction framework 302. Thevision correction framework 302 may be implemented on an imageprocessing device 304 and a spectacles device 306 for placing on asubject. The image processing device 304 may be contained entirely in anexternal image processing device or other computer, while in otherexamples all or part of the image processing device 304 may beimplemented within the spectacles device 306.

The image processing device 304 may include a memory 308 storinginstructions 310 for executing the testing and/or visioning modesdescribed herein, which may include instructions for collectinghigh-resolution images of a subject from the spectacles device 306. Inthe visioning mode, the spectacles device 306 may capture real-timevisual field image data as raw data, processed data, or pre-processeddata. In the testing mode, the spectacles device may project testingimages (such as the letters “text” or images of a vehicle or otherobject) for testing aspects of a visual field of a subject.

The spectacles device 306 may be communicatively connected to the imageprocessing device 304 through a wired or wireless link. The link may bethrough a Universal Serial Bus (USB), IEEE 1394 (Firewire), Ethernet, orother wired communication protocol device. The wireless connection canbe through any suitable wireless communication protocol, such as, WiFi,NFC, iBeacon, Bluetooth, Bluetooth low energy, etc.

In various embodiments, the image processing device 304 may have acontroller operatively connected to a database via a link connected toan input/output (I/O) circuit. Additional databases may be linked to thecontroller in a known manner. The controller includes a program memory,the processor (may be called a microcontroller or a microprocessor), arandom-access memory (RAM), and the input/output (I/O) circuit, all ofwhich may be interconnected via an address/data bus. It should beappreciated that although only one microprocessor is described, thecontroller may include multiple microprocessors. Similarly, the memoryof the controller may include multiple RAMs and multiple programmemories. The RAM(s) and the program memories may be implemented assemiconductor memories, magnetically readable memories, and/or opticallyreadable memories. The link may operatively connect the controller tothe capture device, through the I/O circuit.

The program memory and/or the RAM may store various applications (i.e.,machine readable instructions) for execution by the microprocessor. Forexample, an operating system may generally control the operation of thevision system 300 such as operations of the spectacles device 306 and/orimage processing device 304 and, in some embodiments, may provide a userinterface to the device to implement the processes described herein. Theprogram memory and/or the RAM may also store a variety of subroutinesfor accessing specific functions of the image processing device 304described herein. By way of example, and without limitation, thesubroutines may include, among other things: obtaining, from aspectacles device, high-resolution images of a visual field; enhancingand/or correcting the images; and providing the enhanced and/orcorrected images for display to the subject by the spectacles device306.

In addition to the foregoing, the image processing device 304 mayinclude other hardware resources. The device may also include varioustypes of input/output hardware such as a visual display and inputdevice(s) (e.g., keypad, keyboard, etc.). In an embodiment, the displayis touch-sensitive, and may cooperate with a software keyboard routineas one of the software routines to accept user input. It may beadvantageous for the image processing device 304 to communicate with abroader network (not shown) through any of a number of known networkingdevices and techniques (e.g., through a computer network such as anintranet, the Internet, etc.). For example, the device may be connectedto a database of aberration data.

In some embodiments, system 100 may store prediction models,modification profiles, visual defect information (e.g., indicatingdetected visual defects of a user), feedback information (e.g., feedbackrelated to stimuli displayed to users or other feedback), or otherinformation at one or more remote databases (e.g., in the cloud). Insome embodiments, the feedback information, the visual defectinformation, the modification profiles, or other information associatedwith multiple users (e.g., two or more users, ten or more users, ahundred or more users, a thousand or more users, a million or moreusers, or other number of users) may be used to train one or moreprediction models. In some embodiments, one or more prediction modelsmay be trained or configured for a user or a type of device (e.g., adevice of a particular brand, a device of a particular brand and model,a device having a certain set of features, etc.) and may be stored inassociation with the user or the device type. As an example, instancesof a prediction model associated with the user or the device type may bestored locally (e.g., at a wearable device of the user or other userdevice) and remotely (e.g., in the cloud), and such instances of theprediction model may be automatically or manually synced across one ormore user devices and the cloud such that the user has access to thelatest configuration of the prediction model across any of the userdevices or the cloud. In some embodiments, multiple modificationprofiles may be associated with the user or the device type. In someembodiments, each of the modification profiles may include a set ofmodification parameters or functions to be applied to live image datafor a given context to generate an enhanced presentation of the liveimage data. As an example, the user may have a modification profile foreach set of eye characteristics (e.g., a range of gaze directions, pupilsizes, limbus positions, or other characteristics). As further example,the user may additionally or alternatively have a modification profilefor each set of environmental characteristics (e.g., a range ofbrightness levels of the environment, temperatures of the environment,or other characteristics). Based on the eye characteristics orenvironmental characteristics currently detected, the corresponding setof modification parameters or functions may be obtained and used togenerate the enhanced presentation of the live image data.

Subsystems 112-124

In some embodiments, with respect to FIG. 1A, testing subsystem 122 mayprovide a visual test presentation to a user. As an example, thepresentation may include a set of stimuli. During the presentation (orafter the presentation), testing subsystem 122 may obtain feedbackrelated to the set of stimuli (e.g., feedback indicating whether or howthe user sees one or more stimuli of the set). As an example, thefeedback may include an indication of a response of the user to one ormore stimuli (of the set of stimuli) or an indication of a lack ofresponse of the user to such stimuli. The response (or lack thereof) mayrelate to an eye movement, a gaze direction, a pupil size change, or auser modification of one or more stimuli or other user input (e.g., theuser's reaction or other response to the stimuli). As another example,the feedback may include an eye image captured during the visual testpresentation. The eye image may be an image of a retina of the eye(e.g., the overall retina or a portion thereof), an image of a cornea ofthe eye (e.g., the overall cornea or a portion thereof), or other eyeimage. In some embodiments, testing subsystem 122 may generate one ormore results based on the feedback, such as affected portions of avisual field of the user, an extent of the affected portions, visionpathologies of the user, modification profiles to correct for theforegoing issues, or other results.

In some embodiments, based on feedback related to a set of stimuli(displayed to a user during a visual test presentation) or otherfeedback, testing subsystem 122 may determine light sensitivity,distortions, or other aberrations related to one or more eyes of theuser. In some embodiments, the set of stimuli may include a pattern, andtesting subsystem 122 may cause the pattern to be projected onto one ormore eyes of the user (e.g., using a projection-based wearablespectacles device). As an example, the pattern may be projected onto aretina or a cornea of the user to determine defects affecting the retinaor the cornea. In one use case, the projection pattern can be used toassess correct for dysmorphopsia in age-related macular degeneration andother retinal pathologies. As shown in FIG. 31A, a digital projection ofa pattern 3100 may be projected onto a subject's eye 3102. The patternmay be digitally generated on a projector positioned on an interior of aspectacles device. A digital camera 3104 (e.g., an inward directed imagesensor) may also be positioned on an interior side of the spectaclesdevice to capture an image of the pattern 3100 reflected from the eye3102. For example, the image capture may be captured from the cornealsurface of the eye, as shown in FIG. 32. From the captured image of thepattern 3100, testing subsystem 122 may determine if the pattern looksnormal (e.g., as depicted in FIG. 33) or exhibits anomalies (e.g., asdepicted in FIG. 34 (3101)). The anomalies may be assessed and correctedfor using one of the techniques described herein.

In some embodiments, testing subsystem 122 may cause a set of stimuli tobe displayed to a user, obtain an image of one or more of the user'seyes (e.g., at least a portion of a retina or cornea of the user) asfeedback related to the set of stimuli, and determine one or moremodification parameters or functions to address light sensitivity,distortions, or other aberrations related to the user's eyes (e.g.,lower or higher order aberrations, static or dynamic aberrations, etc.).Such modifications may include transformations (e.g., rotation,reflection, translation/shifting, resizing, etc.), image parameteradjustments (e.g., brightness, contrast, saturation, sharpness, etc.),or other modifications. As an example, when a pattern (e.g., an Amslergrid or other pattern) is projected onto a retina or cornea of the user,the obtained image may include a reflection of the projected patternwith the aberrations (e.g., reflected from the retina or cornea).Testing subsystem 122 may automatically determine the modificationparameters or functions to be applied to the pattern such that, when themodified pattern is projected onto the retina or cornea, an image of theretina or cornea (subsequently obtained) is a version of thepre-modified-pattern image without one or more of the aberrations. Inone use case, with respect FIG. 31C, when the pattern 3100 is projectedonto a retina of the user, the obtained image may include the pattern3100 with distortions (e.g., an inverse of the distortions depicted inmodified pattern 3100′ of FIG. 31D). A function (or parameters for sucha function, e.g., that inverses the distortions in the obtained image)may be determined and applied to the pattern 3100 to generate themodified pattern 3100′. When the modified pattern 3100′ is projectedonto the user's retina, the reflection of the modified pattern 3100′from the user's retina will include the pattern 3100 of FIG. 31C withoutthe prior distortions. To the extent that the reflection still includesdistortions, testing subsystem 122 may automatically update the modifiedparameters or functions to be applied to the pattern to further mitigatethe distortions (e.g., shown in the reflection of the retina).

In another use case, the eye image (e.g., the image of one or more ofthe user's eyes) capturing the projected stimuli (e.g., pattern or otherstimuli) reflected from a retina or cornea may be used to determine afunction (or parameters for the function) to correct for one or moreother aberrations. Upon applying a determined function or parameters tothe projected stimuli, and to the extent that the reflection of themodified stimuli still includes aberrations, testing subsystem 122 mayautomatically update the modified parameters or functions to be appliedto the stimuli to further mitigate the aberrations (e.g., shown in thereflection). In a further use case, the foregoing automateddeterminations of the parameters or functions may be performed for eacheye of the user. In this way, for example, the appropriate parameters orfunctions for each eye may be used to provide correction forAnisometropia or other conditions in which each eye has differentaberrations. With respect to Anisometropia, for example, typicalcorrective glass spectacles cannot correct for the unequal refractivepower of both eyes. That is because the corrective glass spectaclesproduced two images (e.g., one to each eye) with unequal sizes(aniseikonia) and the brain could not fuse those two images into abinocular single vision, resulting in visual confusion. That problem issimply because the lenses of glass spectacles are either convex, magnifythe image or concave, minify the image. The amount of magnification orminification depends on the amount of correction. Given that theappropriate parameters or functions may be determined for each eye, theforegoing operations (or other techniques described herein) can willcorrect for Anisometropia (along with other conditions in which each eyehas different aberrations), thereby avoiding visual confusion or otherissues related to such conditions.

In some embodiments, with respect to FIG. 1A, testing subsystem 122 maycause a set of stimuli to be displayed to a user and determine one ormore modification parameters or functions (to address light sensitivity,distortions, or other aberrations related to the user's eyes) based onthe user's modifications to the set of stimuli or other user inputs. Insome scenarios, with respect to FIG. 31C, the pattern 3100 may be a grid(e.g., an Amsler grid) or any known reference shape designed to allowfor detecting a transformation needed to treat one or more ocularanomalies. That transformation may then be used to reverse-distort theimage in real-time to allow better vision. In an example implementationof FIG. 8, a vision system 800 may include a testing module 802. Thetesting module 802 may be associated with wearable spectacles or may beexecuted in combination with an external device as described elsewhereherein. The testing module 802 may present testing stimuli comprising anAmsler grid to a subject 806. The subject, via the user device 808 orother input device, may manipulate the image of the grid to improvedistortions (e.g., by dragging or moving one or more portions of thelines of the grid). The vision correction framework 810 may present theAmsler grid for further correction by the subject. When the subject hascompleted their manual corrections (e.g., resulting in modified pattern3100′), the vision correction framework 810 may generate themodification profile of the subject to apply to visual scenes when theyare using the spectacles device. As an example, the vision correctionframework 810 may generate an inverse function (or parameters for such afunction) that outputs the modified pattern 3100′ when the pattern 3100is provided as input the function. The described workflow of visionsystem 800 may similarly be applicable to other testing mode operationsdescribed herein.

FIG. 31B is a schematic illustration of the presentation of an Amslergrid 3100 (e.g., an example reference image) displayed as an image on awearable spectacle (e.g., VR or AR headset). The Amsler grid 3100 may bedisplayed to or projected onto a cornea and/or retina of the subject. Anexample standard grid 3100 is shown in FIG. 31C. The same grid patternmay be displayed on a user device. The subject may manipulate the linesof the grid pattern, particularly the lines that appear curved,utilizing a keyboard, mouse, touch screen, or other input on a userdevice, which may include a user interface. The subject can specify ananchor point 3102 from which to manipulate the image. After specifyingthe anchor point, the subject can use the user device (e.g., arrow keys)to adjust the specified line, correcting the perceived distortion causedby their damaged macula. This procedure may be performed on each eyeindependently, providing a set of two modified grids.

Once the subject completes the modification of the lines to appearstraight, a vision correction framework takes the new grids and generatemeshes of vertices corresponding to the applied distortions. Thesemeshes, resulting from the testing mode, are applied to an arbitraryimage to compensate for the subject's abnormalities. For example, eacheye may be shown the modified image corresponding to the appropriatemesh, as part of confirmation of the testing mode. The subject can thenindicate on the user device if the corrected images appear faultlesswhich, if true, would indicate that the corrections were successful. Forexample, FIG. 31E illustrates an actual scene, as it should be perceivedby the user. FIG. 31F illustrates a corrected visual field that whenprovided to a subject with a visual distortion determined by the Amslergrid technique, results in that subject seeing the visual field of FIG.31F as the actual visual field of FIG. 31E.

Such correction may be performed in real time on live images to presentthe subject with a continuously corrected visual scene. The correctionmay be achieved real-time whether the spectacles device includesdisplays that generate the capture visual field or whether thespectacles device is custom-reality based and uses a correction layer toadjust for the distortion, as both cases may utilize the determinedcorrective meshes.

In some examples, a reference image such as the Amsler pattern may bepresented directly on a touch screen or tablet PC, such as 3150 (e.g., atablet PC) shown in FIG. 31G. The Amsler pattern is presented on adisplay of the device 3150, and the subject may manipulate the linesthat appear curved using a stylus 3152 to draw the corrections that areto be applied to the lines to make them appear straight. During thetesting mode, after each modification, the grid may be redrawn toreflect the latest edit. This procedure may be performed on each eyeindependently, providing us a set of two modified grids. After thesubject completes the testing mode modification, the tablet PC executesan application that creates and sends the mesh data to an accompanyingapplication on the spectacles device to process images that apply thedetermined meshes.

Once the spectacles device receives the results of the testing modemodification, the spectacles device may apply them to an arbitrary imageto compensate for the subject's abnormalities. The images that resultfrom this correction may then be displayed. The display may be via anVR/AR headset. In one example, the display presents the images to theuser via the headset in a holographical way. Each displayed image maycorrespond to the mesh created for each eye. If the corrected imagesseem faultless to the subject, the corrections may be consideredsuccessful and may be retained for future image processing. In someembodiments of the testing mode, instead of or in addition to presentinga single image modified according to the modified grids, a videoincorporating the modifications may be presented. In one example, thevideo includes a stream of a camera's live video feed through thecorrection, which is shown to the subject.

In some embodiments, with respect to FIG. 1A, testing subsystem 122 maydetermine one or more defective visual field portions of a visual fieldof a user (e.g., an automatic determination based on feedback related toa set of stimuli displayed to the user or other feedback). As anexample, a defective visual field portion may be one of the visual fieldportions of the user's visual field that fails to satisfy one or morevision criteria (e.g., whether or an extent to which the user senses oneor more stimuli, an extent of light sensitivity, distortion, or otheraberration, or other criteria). In some cases, the set of stimulidisplayed to the user includes at least one testing image of text or ofan object. Defective visual field portions may include regions ofreduced vision sensitivity, regions of higher or lower opticalaberrations, regions of reduced brightness, or other defective visualfield portions. In some cases, the set of stimuli may differ in contrastlevels with respect to each other and with respect to a baselinecontrast level by at least 20 dB. In some cases, the set of stimuli maydiffer in contrast levels with respect to each other and with respect toa baseline contrast level by at least 30 dB. In some cases, testingsubsystem 122 may, in the testing mode, instruct a wearable spectaclesdevice to display the set of testing stimuli to the user in a descendingor ascending contrast.

In one use case, testing was performed on 4 subjects. A testing protocolincluded a display of text at different locations one or more displaymonitors of the spectacles device. To assess the subject's visual fieldof impaired regions, the word “text” was displayed on the spectaclemonitors for each eye, and the subject was asked to identify the “text.”Initially the “xt” part of the word “text” was placed intentionally bythe operator on the blind spot of the subject. All 4 subjects reportedonly seeing “te” part of the word. The letters were then moved usingsoftware to control the display, specifically. The text “text” was movedaway from the blind spot of the subject who was again asked to read theword. Subjects were able to read “text” stating that now the “xt” partof the word has appeared.

An example of this assessment protocol of a testing mode is shown inFIGS. 6A-6C. As shown in FIGS. 6A-6B, the code automatically detects theblind spots on a Humphrey visual field. The word “text” 600 is projectedso that “xt” part of the word is in a blind spot 602 (FIG. 6A). Thesubject was asked to read the word. The word “text” 600 was then movedaway from the blind spot 602 (FIG. 6B) and the subject was asked to readit again. The word “text” 600 can be displayed at different coordinatesof the visual field of the subject, with the visual field divided into 4coordinates in the illustrated example. This protocol allows foridentification of multiple blind spots, including peripheral blind spot604. The text may be moved around over the entire visual field of thesubject, with the subject being asked to identify when all or portionsof the text is not visible or partially visible or visible with areduced intensity.

The pupil tracking functionalities described herein may include pupilphysical condition (e.g., visual axis, pupil size, and/or limbus),alignment, dilation, and/or line of sight. Line of sight, also known asthe visual axis, is a goal that can be achieved by one or more oftracking the pupil, the limbus (which is the edge between the cornea andthe sclera), or even track blood vessel on the surface of the eye orinside the eye. Thus, pupil tracking may similarly include limbus orblood vessel tracking. The pupil tracking may be performed utilizing oneor more inward facing image sensors as described herein. In variousembodiments, pupil tracking functionalities may be used fordetermination of parameters for registering the projected image on thevisual field of the subject (FIG. 6C).

With respect to FIG. 6C, a GUI 606 display may be displayed to anoperator. The GUI 606 may provide information related to the testing.For example, the GUI 606 shows measured visual field defects and therelative location of the image to the defects. The GUI 606 may beoperable to allow automatic distribution of the images to the functionalpart of the visual field but may include buttons to allow the operatorto override the automatic mode. The external image processing device maybe configured to determine where this assessment text is to be displayedand may wirelessly communicate instructions to the digital spectacles todisplay the text at the various locations in the testing mode.

In another use case, with respect to FIGS. 7A-7C, instead of “text”being used, the subject was tested to determine whether they could see acar 700 placed in different portions of the visual field, for pupiltracking and affected region determination. The pupil trackingfunctionality allows the vision system to register the projected imageon the visual field of the subject.

In some embodiments, with respect to FIG. 1A, testing subsystem 122 maydetermine one or more defective visual field portions of a visual fieldof a user based on a response of the user's eyes to a set of stimulidisplayed to the user or lack of response of the user's eyes to the setof stimuli (e.g., eye movement response, pupil size response, etc.). Insome embodiments, one or more stimuli may be dynamically displayed tothe user as part of a visual test presentation, and the responses orlack of responses to a stimulus may be recorded and used to determinewhich part of the user's visual field is intact. As an example, if aneye of the user responds to a displayed stimulus (e.g., by changing itsgaze direction toward the displayed stimulus's location), the eye'sresponse may be used as an indication that the eye can see the displayedstimulus (e.g., and that a corresponding portion of the user's visualfield is part of the user's intact visual field). On the other hand, ifan eye of the user does not respond to a displayed stimulus (e.g., itsgaze direction does not move toward the displayed stimulus's location),the eye's lack of response may be used as an indication that the eyecannot see the displayed stimulus (e.g., and that a correspondingportion of the user's visual field is a defective visual field portion).Based on the foregoing indications, testing subsystem 122 mayautomatically determine the defective visual field portions of theuser's visual field.

In some embodiment, the set of stimuli displayed to the user may includestimuli of different brightness, contrast, saturation, or sharpnesslevels, and the responses or lack of responses to a stimulus having aparticular brightness, contrast, saturation, or sharpness level mayprovide an indication of whether a portion of the user's visual field(corresponding to the location of the displayed stimuli) has an issuerelated to brightness, contrast, saturation, or sharpness. As anexample, if an eye of the user responds to a displayed stimulus having acertain brightness level, the eye's response may be used as anindication that the eye can see the displayed stimulus (e.g., and that acorresponding portion of the user's visual field is part of the user'sintact visual field). On the other hand, if an eye of the user does notrespond to a stimulus having a lower brightness level (e.g., that anormal eye would respond to) at the same location, the eye's lack ofresponse may be used as an indication that a corresponding portion ofthe user's visual field has reduced brightness. In some cases, thebrightness level for the stimulus may be incrementally increased untilthe user's eye responds to the stimulus or until a certain brightnesslevel threshold is reached. If the user's eye eventually reacts to thestimulus, the current brightness level may be used to determine a levelof light sensitivity for that corresponding virtual field portion. Ifthe brightness level threshold is reached and the user's eye does notreact to the stimulus, it may be determined that the correspondingvirtual field portion is a blind spot (e.g., if the correspondingchanges to one or more of contrast, saturation, sharpness, etc., to thestimulus also does not trigger an eye response). Based on the foregoingindications, testing subsystem 122 may automatically determine thedefective visual field portions of the user's visual field.

In some embodiment, a fixation point for a visual test presentation maybe dynamically determined. In some embodiments, a location of a fixationpoint and locations of the stimuli to be displayed to the user may bedynamically determined based on gaze direction or other aspect of theuser's eyes. As an example, during a visual test presentation, both thefixation points and stimuli locations are dynamically represented to apatient relative to the patient's eye movement. In one use case, thecurrent fixation point may be set to a location of the visual testpresentation that the patient is currently looking at a particularinstance, and a test stimulus may be displayed relative to that fixationpoint. In this way, for example, the patient is not required to fix hisattention to a certain predefined fixation location. This allows thevisual test presentation to be more objective, interactive, and reducestress caused by prolonged fixation on a fixed point. The use of dynamicfixation points also eliminates patient errors related to fixationpoints (e.g., if the patient forgets to focus on a static fixationpoint).

In some embodiments, the fixation point may be locked, and one or moretest stimuli may be displayed relative to that fixation point until thelock is released (e.g., FIG. 35F). Upon the lock being released, thecurrent fixation point may be set to a location of the visual testpresentation that the patient is currently looking at a particularinstance. The new fixation point may then be locked, and one or moresubsequent test stimuli may be displayed relative to that new fixationpoint. In some embodiments, while the fixation point remains the same,multiple stimuli may be displayed at one or more different locations onthe visual test presentation. As an example, as the fixation pointremains the same, one or more stimuli may be displayed after one or moreother stimuli are displayed. In some embodiments, each of the multiplestimuli may be displayed and then deemphasized on or removed from theuser interface on which the visual test presentation is provided. As anexample, as the fixation point remains the same, one or more stimuli maybe displayed and deemphasized/removed after one or more other stimuliare displayed and deemphasized/removed. In one use case, the brightnessor other intensity level of a stimulus may be decreased (e.g., decreasedby a predefined amount, decreased to a default “low” threshold level,decreased to a personalized threshold level at which it has beendetermined that the patient cannot see, etc.) to perform deemphasis ofthe stimulus. In another use case, the stimulus may be removed from theuser interface (e.g., the stimulus is no longer being displayed by theuser interface).

As discussed, in some embodiments, testing subsystem 122 may adjust afixation point (e.g., for a visual test presentation) based on eyecharacteristic information related to a user (e.g., a patient's eyemovement, gaze direction, or other eye-related characteristics, such asthose occurring during the visual test presentation). In one use case,testing subsystem 122 may cause a first stimulus to be displayed at afirst interface location on a user interface (e.g., of a wearable deviceor other device of the user) based on the fixation point. Testingsubsystem 122 may adjust the fixation point based on the eyecharacteristic information and cause a second stimulus to be displayedat a second interface location on the user interface during the visualtest presentation based on the adjusted fixation point. As discussed, insome embodiments, one or more stimuli may be displayed on the userinterface (e.g., at different interface locations) between the displayof the first stimulus and the display of the second stimulus. Testingsubsystem 122 may obtain feedback information during the visual testpresentation and generate visual defect information based on suchfeedback information. As an example, the feedback information mayindicate feedback related to the first stimulus, feedback related to thesecond stimulus, feedback related to a third stimulus displayed duringthe visual test presentation, or feedback related to one or more otherstimuli. Such feedback may indicate (i) a response of the user to astimulus, (ii) a lack of response of the user to a stimulus, (iii)whether or an extent to which the user senses one or more stimuli, anextent of light sensitivity, distortion, or other aberration, or (iv)other feedback. The generated visual defect information may be used to(i) train one or more prediction models, (ii) determine one or moremodification profiles for the user, (iii) facilitate live imageprocessing to correct or modify images for the user, (iv) or performother operations described herein.

In some embodiments, the use of a dynamic fixation point during a visualtest presentation may facilitate greater coverage of a user's visualfield than the dimensions of a view provided via a user interface. As anexample, as indicated with respect to FIGS. 35A-35E, the user interface(e.g., of a wearable device or other device of the user) may beconfigured to display a view having one or more dimensions, where eachof the dimensions correspond to a number of degrees (e.g., a width of 70degrees, a height of 70 degrees, a width or height of another number ofdegrees, etc.). Through the use of the dynamic fixation point, however,testing subsystem 122 may generate visual defect information havingcoverage greater than the degrees with respect to one or more of thedimensions (e.g., a horizontal dimension for the user's visual fieldcompared to the width of the user interface view, a vertical dimensionfor the user's visual field compared to the height of the user interfaceview, etc.). In one scenario, based on such techniques, the visualdefect information may have coverage for up to a 2.85 times larger areathan the overall user interface view, and the coverage area may beincreased to a size that approaches 4 times the overall user interfaceview (e.g., if the distance between the wearable device and the eye ofthe user decreases or if the distance between two monitors of thewearable device increases). In addition, the visual defect informationmay have coverage for up to twice the width of the user's visual fieldarea than the width of the user interface view, up to twice the heightof the user's visual field area than the height of the user interfaceview, or other expanded area of the user's visual field. In anotherscenario, based on such techniques, the visual defect information mayindicate whether or the extent to which defects exist at two or morevisual field locations of the user's visual field, where the visualfield locations are apart from one another by greater than the degreesof the user interface view dimensions with respect to one or more of thedimensions for the user's visual field.

In one use case, with respect to FIG. 35A, the use of a dynamic fixationpoint and user interface 3502 (e.g., which is configured to provide a70-degree view) may facilitate the generation of a visual field map 3504that has coverage greater than 70 degrees in both the horizontal andvertical dimensions. As an example, as indicated in FIG. 35A, stimulus3506 a may be displayed at a center of user interface 3502 to cause theuser to look at the center of user interface 3502 to initially set thefixation point 3508 to the center of user interface 3502. Specifically,when the user's eye-related characteristics (e.g., as detected by eyetracking techniques described herein) indicate that the user looking atstimulus 3506 a, the fixation point 3508 for the visual testpresentation may currently be set to the location of user interface 3502corresponding to stimulus 3506 a. In some use cases, the fixation point“floats” on the user interface 3502 in accordance with where the user iscurrently looking.

In a further use case, as indicated in FIG. 35B, stimulus 3506 b may bedisplayed at the bottom left-hand corner of user interface 3502 (e.g.,50 degrees away from the location of user interface 3502 at whichstimulus 3506 a was displayed). If the user's eye-relatedcharacteristics indicate that the user senses stimulus 3506 b (e.g., theuser's eye movement is detected as being toward stimulus 3506 b), visualfield map 3504 may be updated to indicate that the user is able to seethe corresponding location in the user's visual field (e.g., 50 degreesaway from the location of the fixation point in the visual field map3504 in the same direction). When the user's eye-related characteristicsindicate that the user is currently looking at stimulus 3506 b, thefixation point 3508 for the visual test presentation may then be set tothe location of the user interface corresponding to stimulus 3506 b.

In another use case, as indicated in FIG. 35C, stimulus 3506 c may bedisplayed at the top right-hand corner of user interface 3502 (e.g., 100degrees away from the location of user interface 3502 at which stimulus3506 b was displayed). If the user's eye-related characteristicsindicate that the user senses stimulus 3506 c, visual field map 3504 maybe updated to indicate that the user is able to see the correspondinglocation of the user's visual field (e.g., hundred degrees away from thelocation of the fixation point in the visual field map 3504 in the samedirection). When the user's eye-related characteristics indicate thatthe user is currently looking at stimulus 3506 c, the fixation point3508 for the visual test presentation may then be set to the location ofuser interface 3502 corresponding to stimulus 3506 c. As indicated inFIG. 35D, stimulus 3506 d may be displayed at the bottom left-handcorner of user interface 3502 (e.g., 100 degrees away from the locationof user interface 3502 at which stimulus 3506 c was displayed). If theuser's eye-related characteristics indicate that the user sensesstimulus 3506 d, visual field map 3504 may be updated to indicate thatthe user is able to see the corresponding location of the user's visualfield (e.g., 100 degrees away from the location of the fixation point inthe visual field map 3504 in the same direction). When the user'seye-related characteristics indicate that the user is currently lookingat stimulus 3506 d, and the fixation point 3508 for the visual testpresentation may then be set to the location of user interface 3502corresponding to stimulus 3506 d. As indicated in FIG. 35E, stimulus3506 e may be displayed to the left of the top right-hand corner of userinterface 3502 (e.g., 90 degrees away from the location of userinterface 3502 at which stimulus 3506 d was displayed). If the user'seye-related characteristics indicate that the user senses stimulus 3506e, visual field map 3504 may be updated to indicate that the user isable to see the corresponding location of the user's visual field (e.g.,90 degrees away from the location of the fixation point in the visualfield map 3504 in the same direction). When the user's eye-relatedcharacteristics indicate that the user is currently looking at stimulus3506 b, the fixation point 3508 for the visual test presentation maythen be set to the location of user interface 3502 corresponding tostimulus 3506 e. In this way, for example, even though the userinterface view was only 70 degrees in both the horizontal and verticaldimensions, the visual field map 3504 currently has coverage for 200degrees of the user's visual field diagonally, 140 degrees of the user'svisual field with respect to the horizontal dimension, and 140 degreesof the user's visual field with respect to the vertical dimension.

In another use case, with respect to FIG. 35B, if the user's eye-relatedcharacteristics indicates that the user did not see stimulus 3506 b(e.g., there was no significant eye movement response to the display ofstimulus 3506 b, the user's gaze did not shift to an area proximate thelocation of stimulus 3506 b on user interface 3502, etc.), visual fieldmap 3504 may be updated to indicate that the user cannot see thecorresponding location in the user's visual field. As such, in somescenarios, the visual field map may indicate vision defects and theircorresponding locations in the user's visual field for an area greaterthan the size of the view of user interface 3502. As an example, evenwhere the user interface view is only 70 degrees in the horizontal andvertical dimensions, the visual field map may indicate vision defects atvisual field locations that are apart from one another by more than 70degrees in each of the horizontal and vertical dimensions (e.g., thedistances between such indicated visual defects may be up to 140 degreesapart with respect to the horizontal dimension, up to 140 degrees apartwith respect to the vertical dimension, etc.).

In some embodiments, to facilitate greater coverage of a user's visualfield (e.g., despite limitation of hardware/software components relatedto the user interface view), one or more locations on a user interfacemay be selected to display one or more stimuli based on the interfacelocations being farther from the current fixation point (e.g., for avisual test presentation). In some embodiments, testing subsystem 122may select a first interface location on the user interface based on thefirst interface location being farther from the fixation point than oneor more other interface locations on the user interface and cause afirst stimulus to be displayed at the first interface location. In someembodiments, after the fixation point is adjusted (e.g., based on theuser's eye-related characteristics), testing subsystem 122 may select asecond interface location on the user interface based on the secondinterface location being farther from the adjusted fixation point thanone or more other interface locations on the user interface and cause asecond stimulus to be displayed at the second interface location.

As an example, the first stimulus may be selected to be added to a queueof stimuli to be displayed (e.g., a queue of stimuli to be displayednext) during the visual test presentation based on (i) the firststimulus being associated with a first visual field location of theuser's visual field and (ii) the first visual field locationcorresponding to the first interface location (e.g., as determined bythe fixation point and the location of the first visual field locationrelative to the fixation point). As further example, the second stimulusmay be selected to be added to the queue during the visual testpresentation based on (i) the second stimulus being associated with asecond visual field location of the user's visual field and (ii) thesecond visual field location corresponding to the second interfacelocation. By selecting “farther” stimuli/locations to be displayed next,testing subsystem 122 adjusts the fixation point farther away from thecenter of the user interface view, thereby increasing the coverage ofthe user's visual field. In one use case, with respect to FIG. 35B,stimulus 3506 b and its corresponding location on user interface 3502are selected to be the next stimulus/location to be displayed during thevisual test presentation as a result of a determination that thecorresponding interface location is one of the farthest locations on theuser interface from the fixation point (located at the center of userinterface 3502). In doing so, the fixation point is adjusted to thebottom left-hand corner of user interface 3502 (e.g., by causing theuser to look there), thereby enabling the next stimulus to be displayedas far as 100 degrees away from the fixation point (e.g., the distancebetween stimulus 3506 b and stimulus 3506 c in FIG. 35C).

In some embodiment, one or more locations of a user's visual field maybe included as part of a set of visual field locations to be testedduring a visual test presentation. As an example, the test set of visualfield locations may be represented by stimuli during the visual testpresentation, and the determination of whether or the extent to whichthe user has visual defects at one or more of the visual field locationsof the test set is based on whether or the extent to which the usersenses one or more of the corresponding stimuli. In some embodiments, avisual field location may be removed from the test set based on adetermination that the visual field location has been sufficientlytested (e.g., by displaying a stimulus at a corresponding location on auser interface and detecting whether or the extent to which the usersenses the displayed stimulus). As an example, the removal of the visualfield location may include labeling the visual field location in thetest set as no longer being available to be selected from the test setduring the visual test presentation. As such, in some scenarios, stimulicorresponding to the removed visual field location may not besubsequently displayed during the visual test presentation, and stimulicorresponding to one or more other visual field locations in the testset may be subsequently displayed during the visual test presentation.In further scenarios, the visual field location may subsequently beadded to the test set (e.g., by labeling the visual field location inthe test as being available to be selected during the visual testpresentation, by removing the prior label specifying that the visualfield location was not available to be selected during the visual testpresentation, etc.).

In some embodiments, where a fixation point has been adjusted to a firstuser interface location on a user interface at which a first stimulus isdisplayed during a visual test presentation, testing subsystem 122 maycause one or more stimuli to be displayed on the user interface based onthe fixation point at the first interface location. Testing subsystem122 may also subsequently cause a second stimulus to be displayed at asecond interface location on the user interface. As an example, thesecond stimulus may be displayed while the fixation point remains at thefirst interface location (e.g., the fixation point may be locked to thefirst interface location until just prior to the second stimulus beingdisplay, until the second stimulus is displayed, etc.). In someembodiments, testing subsystem 122 may detect that an eye of the userhas fixated on the second interface location based on eye characteristicinformation related to the user, and testing subsystem 122 may adjustthe fixation point to the second interface location based on thefixation detection.

In some embodiments, testing subsystem 122 may establish a lock of afixation point for a visual test presentation to prevent adjustment (orreadjustment) of the fixation point to a different interface location onthe user interface while the lock remains established. In this way, forexample, while the lock of the fixation point remains established, oneor more stimuli may be displayed on the user interface to test one ormore locations of the user's the visual field relative to the lockedfixation point. Subsequently, when the lock of the fixation point isreleased, the fixation point may again be dynamically adjusted. As anexample, testing subsystem 122 may cause a stimulus to be presented at anew interface location (different from the interface location to whichthe fixation point was set) on the user interface. Based on detectingthat an eye of the user has fixated on the new interface location, andafter the lock of the fixation point is released, testing subsystem 122may adjust the fixation point to the new interface location. In one usecase, as discussed above with respect to FIG. 35F, the fixation pointlock may be released to allow the user to “catch” the stimulus (e.g.,operation 3544), and the fixation point lock may then be reinstated tothe new interface location based on the user looking at the stimulus(e.g., operation 3546). Specifically, if the user has “caught” thestimulus (and is still looking at the stimulus), the location of thestimulus becomes the new fixation point.

In some embodiments, while a fixation point remains at a first interfacelocation on a user interface, testing subsystem 122 may cause multiplestimuli to be displayed at interface locations different from the firstinterface location. As an example, subsequent to one or more stimuli ofthe multiple stimuli being displayed on the user interface, one or moreother stimuli of the multiple stimuli may be displayed on the userinterface. As another example, a stimulus may be displayed on the userinterface and then deemphasized on or removed from the user interface,and another stimulus may be subsequently displayed on the user interfaceand deemphasized on or removed from the user interface. In one use case,with respect to FIG. 35F, the fixation point may be locked to aninterface location (at which a prior stimulus was displayed) (e.g.,operation 3546), and one or more stimuli may be displayed on the userinterface at a new interface location (e.g., operation 3528, operation3540 a, etc.) based on the fixation point.

In another use case, multiple locations of the user's visual field maybe tested by displaying multiple stimuli at different interfacelocations while the fixation point remains locked. As an example, withrespect to FIG. 35C, the fixation point may instead be locked to theinterface location at which stimulus 3506 b is displayed on userinterface 3502, and the portion of the user's visual field correspondingto the top right hand corner of visual field map 3504 may be tested bydisplaying stimuli at different locations of user interface 3502 whilethe fixation point remains locked at the interface location of stimulus3506 b.

In some embodiment, one or more interface locations of a user interfacemay be predesignated to be a fixation point relative to which a user'svisual field is tested. As an example, where the four corners of a userinterface are predesignated to each be a fixation point during a visualtest presentation, testing subsystem 122 may initially cause a stimulusto be displayed at the center of the user interface so that the userwill initially fixate on the center stimulus (e.g., the initial fixationpoint). Testing subsystem 122 may then cause a stimulus to be displayedat the top right-hand corner of the user interface and, upon detectingthat the user sees the top right stimulus (e.g., based on eyecharacteristics of the user), adjust and lock the fixation point to thetop right-hand corner of the user interface. Testing subsystem 122 maysubsequently test a portion of the user's visual field by causingstimuli to be displayed at different locations of on the user interfacewhile the fixation point remains locked. In one use case, if the userinterface is represented by user interface 3502 of FIG. 35A, and theuser's visual field is represented by visual field map 3504 of FIG. 35A,by displaying the stimuli at different locations of the user interfacewhile the fixation point remains locked to the top right-hand corner,the portion of the user's visual field corresponding to the bottomleft-hand quarter of visual field map 3504 may be thoroughly tested. Theforegoing process may then be repeated for the other corners of the userinterface to test the portions of the user's visual field correspondingto the other parts of visual field map 3504.

In some embodiments, while a fixation point remains at a first interfacelocation on a user interface (at which a first stimulus is displayed),testing subsystem 122 may cause multiple stimuli to be displayed andthen deemphasized on or removed from the user interface while the firststimulus continues to be displayed at the first interface location onthe user interface. As an example, where the first interface location isthe top right-hand corner of the user interface, the first stimulus maycontinue to be displayed while a series of other stimuli are momentarilydisplayed on the user interface. As such, the visual change occurring atanother interface location (from another stimulus appearing at thatother interface location) will cause the user to look at the source ofthe visual change if the other interface location does not correspond toa defective portion of the user's visual field (e.g., a blind spot ofthe user's visual field). However, when the other stimulus disappears,the user will fixate back on the top right-hand corner because the firststimulus will be the primary (or only) source of visual simulation forthe eye of the user.

In some embodiments, while a fixation point remains at a first interfacelocation on a user interface (at which a first stimulus is displayed),testing subsystem 122 may cause the first stimulus to be deemphasized onor removed from the user interface and then emphasized or redisplayed atthe first interface location on the user interface. In some embodiments,while the fixation point remains at the first interface location,testing subsystem 122 may cause multiple stimuli to be displayed on theuser interface and, subsequent to the display of at least one stimulusof the multiple stimuli, cause the first stimulus to be emphasized orredisplayed at the first interface location on the user interface. Inone use case, if the brightness of the first stimulus was decreased, thebrightness of the first stimulus may be increased so that the eye of theuser will detect the visual change (and the increased visualstimulation) and fixate back on the first interface location at whichthe first stimulus is displayed on the user interface. In another usecase, if the first stimulus was removed from the user interface, theredisplay of the first stimulus will likewise cause the eye of the userto fixate back on the first interface location on the user interface.

In some embodiments, one or more portions of the process shown in FIG.35F may be used to facilitate a visual test presentation using a dynamicfixation point. With respect to FIG. 35F, in operation 3522, a matrix ofpossible stimuli (e.g., all possible stimuli) in a visual field of auser is created or obtained. In operation 3524, an eye tracking deviceis used to lock a floating fixation point is locked to the center of thevisual field. As an example, the eye coordinates obtained from the eyetracking device may be used to “float” the floating fixation pointaround with the eye. In operation 3526, the available stimuli of thematrix may be ranked (e.g., so that the farthest point from the floatingfixed point is first). As an example, stimuli corresponding to locationsof the user interface view that are at least as far from the fixationpoint as all other locations on the user interface (that correspond toan available stimulus of the matrix) may be ranked ahead of all theother available stimuli (or ranked with equal priority as other stimulithat are of equal distance from the floating fixation point). As anexample, the ranking may be performed in real-time using the eyetracking device (e.g., pupil or eye tracker or other eye trackingdevice).

In operation 3528, subsequent to the ranking, the first stimulus on theranking list (e.g., the stimulus with the highest priority) may the nextstimulus to be displayed during the visual test presentation. As anexample, the stimulus may be displayed in a color that highly contrastswith the background (e.g., the stimulus color may be black to contrast ablack background). In operation 3530, eye movement vectors (or otherrepresentation of eye-related characteristics) may be consistentlymeasured using the eye tracking device. If eye movement is not detectedto be toward the stimulus (operation 3532), then, in operation 3534, thestimulus is counted as not being seen and will be removed from thematrix of available stimuli. Operations 3528-3530 will be repeated withthe current highest ranked stimulus on the ranking list (that is in thematrix of available stimuli).

If eye movement is detected to be toward the stimulus (operation 3536)(e.g., thereby, indicating that the user senses the stimulus), then, inoperation 3538, the stimulus is counted as being seen (qualitatively),and the stimulus disappears from the user interface. In operations 3540a-3540 d, the visual test presentation may test the extent to which theuser can sense a stimulus in the particular area of the visual field. Asan example, in operation 3540 a, the stimulus appears back in a colorshade (e.g., grey shade) that gets darker every time this operation isexecuted. In one use case, the stimulus may initially appear back in acolor that is similar to the background color (e.g., the stimulus colormay initially be a light grey color when the background color is white).In operation 3540 b, eye movement vectors (or other representation ofeye-related characteristics) may be constantly measured using the eyetracking device. If eye movement is not detected to be toward thestimulus (operation 3540 c), then the operations 3540 a and 3540 b willbe repeated (e.g., with a darker color shade to further contrast thewhite background color). If eye movement is detected to be toward thestimulus, the sensitively of vision is indicated for the particular areaof the visual field based on the degree of the color shade (e.g., thedegree of the grey shade) of the displayed stimulus (operation 3542).

In operation 3544, the eye tracking/floating fixation point lock isreleased (e.g., to allow the user to catch the stimulus). In operation3546, the eye tracking/floating fixation point lock is reinstated (e.g.,based on where the user is currently looking). As an example, if theuser has “caught” the stimulus (and is still looking at the stimulus),the location of the stimulus becomes the new floating fixation point. Inoperation 3548, the stimulus is removed from the matrix of availablestimuli, and the process repeats with operation 3526 with respect to theother available stimulus of the matrix.

In some embodiments, testing subsystem 122 may determine or confirm thata user sees a stimulus (and, thus, has vision in a corresponding visualfield location) on a user interface based on a determination that theuser's eye movement is an intentional movement toward the stimulus. Insome embodiments, one or more bounding boxes or other structuresdefining a region of a user interface may be used to determine orconfirm that a user can see a stimulus. In some embodiments, the definedregion may be a region within which a user's eye (e.g., the user's gazedirection) must remain while moving toward a displayed stimulus. In someembodiments, the defined region may be a region between lines equal toor less than a certain number of degrees (e.g., 35 degrees, 30 degrees,25 degrees, 20 degrees, 15 degrees, 10 degrees, or other number ofdegrees) away from a straight, direct path from the fixation point tothe stimulus. As an example, if testing subsystem 122 determines thatthe user's eye movement remained within the defined region when thestimulus is displayed until the user's eye has reached the particularlocation of the user interface at which the stimulus is displayed, suchdetermination may be used as a confirmation that the user actually sawthe stimulus. In one use case, with respect to FIG. 35G, when the user'seye is viewing user interface 3562 and fixated at fixation point 3564,stimulus 3566 may be presented near the top-right corner of userinterface 3562. If the user's eye movement stays within region 3568(e.g., between the boundary lines forming a cone-shaped boundarystructure) until the eye reaches the location of the user interface atwhich stimulus 3566 is presented (e.g., as shown by gaze travel 3570),testing subsystem 122 may determine with high accuracy that the useractually saw stimulus 3566. In this way, for example, the amount of timerequired to detect that the user has seen may be significantly reducedvia use of such determination. For example, the use of eye trackerstypically requires additional confirmation time (e.g., ˜400 ms for a 30Hz eye tracker) to detect that an eye is fixating on a stimulusposition. Through the use of such a bounding structure (or such regiondefined by the bounding structure), the foregoing additionalconfirmation time may be eliminated for each stimulus to be tested.

In addition, eye trackers are generally more accurate for tracking eyemovement or fixation in a central region (e.g., the macular region ofthe user's visual field), but less accurate for tracking eye movementsor fixation in the periphery. Thus, in some embodiments, such technicaldeficiencies of eye trackers with respect to tracking peripheral eyemovements or fixation may be overcome via the foregoing use of thedefined region. As indicated above, if testing subsystem 122 determinesthat user's eye movement remained within the defined region when thestimulus is displayed until the eye tracker has detected that the user'seye has reached the particular location of the user interface at whichthe stimulus is displayed, such determination may be used as aconfirmation that the user actually saw the stimulus. In this way, forexample, despite the greater error rate of eye trackers for trackingperipheral eye fixation, the confirmation from the foregoingdetermination may be used to decrease such errors.

As another example, if testing subsystem 122 determines that the user'seye moved outside of a defined region (e.g., defined by a boundingstructure) when a stimulus was displayed, such determination may be usedto retest the particular location of the user interface at which thestimulus was displayed, invalidate an initial determination that theuser can see the displayed stimulus at the particular location, orindicate that the user cannot see the displayed stimulus at theparticular location. In one use case, with respect to FIG. 35H, when theuser's eye is viewing user interface 3562 and fixated at fixation point3564, the user may not see stimulus 3566 when it is presented near thetop-right corner of user interface 3562 (e.g., due to a visual fielddefect at the corresponding location of the user's visual field).Because the user does not see stimulus 3566, the user's eye may start towonder around. In some cases, however, by wondering around, the user'seye may unintentionally land on stimulus 3566, which may cause a typicalvision testing system to determine that the user can see stimulus 3566.However, in this case, because the user's eye movement went outside ofregion 3568 (e.g., as shown by gaze travel 3572), testing subsystem 122may determine that the user did not see stimulus 3566 or retest thecorresponding location (e.g., by increasing the contrast level oradjusting another characteristic of stimulus 3566 at another time duringthe visual test) even if the user's eye eventually lands on stimulus3566.

In some embodiments, testing subsystem 122 may determine or confirm thata user can see a stimulus based on a determination that the user's eyeremained within a defined region (e.g., defined by a bounding structure)when the stimulus is displayed until the user's eye has moved at least athreshold distance toward the stimulus (or toward the location of theuser interface at which the stimulus is displayed). As an example, withrespect to FIG. 35I, when the user's eye is fixated at fixation point3564, stimulus 3566 may be presented near the top-right corner of userinterface 3562. If the user's eye movement stays within region 3568(e.g., between the boundary lines forming a cone-shaped boundarystructure) until the eye passes line 3574 (e.g., as shown by gaze travel3576), testing subsystem 122 may determine that the user's eye movementwas an intentional movement toward stimulus 3566 (and, thus, that theuser saw stimulus 3566). As an example, line 3574 may represent 60% ofthe distance between the fixation point and location of the userinterface at which stimulus 3566 is presented or other percentage ofsuch distance (e.g., 50%, 70%, 80%, 90%, etc.). In this way, forexample, the amount of time required to detect that the user has seen astimulus may further be reduced via use of such determination. Forexample, the wait time for the eye movement may be ˜200-400 ms (e.g.,depending on how peripheral the stimulus is). Through the use of such abounding structure (or such region defined by the bounding structure),accurate detection may be achieved without waiting for the eye to landon each stimulus, thereby reducing the wait time because the eye wouldnot need to travel all the way to the location at which the stimulus ispresented. Once it is determined that the eye movement is an intentionalmovement toward the stimulus, the vision test may move on to testing oneor more other locations.

In some embodiments, the location of a fixation point or the locationsof the stimuli to be displayed to the user may be static during a visualtest presentation. As an example, testing subsystem 122 may display astimulus in the center of the user interface (or the locationcorresponding to the static fixation point) to cause the user to look atthe center of the user interface (or other such location correspondingto the static fixation point). Once the user is detected as looking atthe static fixation point location, testing subsystem 122 may displaythe next stimulus of a set of stimuli for testing one or more areas ofthe user's visual field. Each time that the user is detected as notlooking at the static fixation point location, testing subsystem 122 mayrepeat the display of a stimulus at the static fixation point location.

As another example, with respect to FIG. 35J, a visual test presentationapplying a fast thresholding strategy may utilize four contrastingstaircase stimuli covering the central 40 degrees' radius using 52stimuli sequences at predetermined locations. In other examples,different numbers of contrast stimuli, coverage, and stimuli locationsmay be used. In this example, the stimuli was located at the center ofeach cell shown in the FIG. 35J. The twelve corner cells, where thestimuli are not visible because of the circular display's lens, were nottested. The spacing between each stimulus location was approximately 10degrees apart. Each stimuli sequence contained four consecutive stimuliat different contrast levels with respect to the background. Stimulicontrast ranged between 33 dB down to 24 dB in steps of 3 dB in adescending order between each contrast level. Threshold values wererecorded at the last seen stimulus. If the patient did not see anystimulus contrast at a specific location, the location is marked unseenand was given a value of 0 dB.

The background had a bright illumination (100 lux) while the stimuliwere dark dots with different contrast degrees. Therefore, the test wasa photopic test rather than a mesopic one. In some embodiments, thebackground may be dark, and the stimuli may comprise bright illuminationdots. Each stimulus was presented for a time period of approximately 250msec, followed by a response waiting time period of approximately 300msec. These time periods were also made adjustable through a controlprogram according to the subject's response speed, which, for example,may be adjusted prior to testing based on pre-test demonstration ordynamically during testing. Generally, a stimulus size of 0.44 degreeswas used at the central 24 degrees' radius, which is equivalent to thestandard Goldmann stimulus size III. The stimulus size at the periphery(between 24 and 40 degrees' radius) was doubled to be 0.88 degrees. Thepurpose of doubling the stimulus size in the peripheral vision was toovercome the degraded display lens performance at the periphery. Thislens degradation effect was significant, as the normal human vision'sacuity even deteriorates at the peripheral regions. The testing programalso had the ability for the stimulus size to be changed for thedifferent patient cases.

The fixation target (pattern) of FIG. 35J was located in the center ofthe screen for each eye tested. This target was designed as a multicolorpoint, rather than a unicolor fixation point as routinely used in thetraditional Humphrey tests. This color changing effect helped grab theattention of the subject and made target focusing easier for them. Thefrequency of the color changes was asynchronous with the stimulusappearance, so that the subject would not relate both events togetherand falsely responds. The testing protocol also had the ability for thefixation target size to be changed according to the patient's condition.In addition, the eye/pupil tracking system may be used to check thesubject's eye fixation at different time intervals. The eye trackingsystem transmits to the testing program the gaze vectors' direction,which informs the program if the subject is properly focused to thecenter or not.

Fixation checks were performed using the pupil/gaze data for each eyeindividually. Pupil/gaze data were acquired at different time instancesand, if the gaze direction vectors were at approximately 0 degrees, thenthe subject is focusing on the center target, otherwise the programwould pause waiting for fixation to restored. If the patient were out offixation, no stimulus was shown and the test was halted until theparticipant gets back in fixation. Offset tolerance was allowed forminor eye movements at the fixation target. Fixation checks wereperformed for each stimuli's location at mainly two time events; beforeshowing each stimulus in the stimuli sequence (e.g., prior to eachstimulus contrast level of the four levels mentioned earlier), andbefore recording a response, whether the response was positive (e.g.,patient saw the stimulus) or negative (e.g., patient did not see thestimulus). Negative responses were recorded at the end of the stimulisequence interval in addition to the allowed response time. Checkingfixation before showing the stimuli sequence was to ensure the patientwas focusing on the fixation target. If the subjects were out offixation, no stimulus was shown, and the test was halted until theparticipant gets back in fixation.

FIG. 36 shows a timing diagram showing operations of a testing sequenceat one stimulus location. In one example, a pupil tracking device, whichmay be separate or a component of a vision system or device thereof, mayinclude inward directed image sensors and be configured to provide datainstructing the image display device, which may include a projector, tochange the location of the stimulus being projected according to line ofsight movement. In this way, even if the subject is looking around andnot fixating, the stimuli may move with the eyes of the subject and willcontinue testing the desired location of the visual field. Therefore,rather than halting the stimuli sequence when the subject is determinedto be focused outside of the fixation target, the stimuli sequence maycontinue with a modification of the stimuli to correspond with theintended location within the subject's visual field within the sequencesas repositioned based on a determination of the subject's currentfixation point.

For each subject, the visual field test started by orienting the subjectof how the test goes. The spectacles device was fitted on the patient toensure that the subject could see the fixation target clearly, and ifnecessary, target size was adjusted accordingly. Eye trackingcalibration was performed at one point, the fixation target. Followingthat, a demonstration mode was presented to the subject. This modefollows the same sequence as the main test, but with only fewerlocations, seven locations in this instance, and without recording anyresponses. The purpose of this mode was to train the subject on thetest. Additionally, this training mode helps the program operator tocheck for the eye tracking system accuracy, patient response speed, andthe patient eye's location with respect to the mounted headset, to makesure that no error or deviation would occur during the full test.

Normal blind spots were then scanned for, by showing suprathresholdstimuli at four different locations spaced by 1 degree in the 15-degreevicinity. This step was beneficial to avoid rotational misfits betweenthe headset and the subject's eyes.

Next, the 52 stimuli sequences were presented to the patient at thepre-specified locations with random order. The subject indicatedresponses by either actuating an electronic clicker or gesturing inresponse to a stimuli. After recording the subject's responses at alllocations, the “unseen” points' locations were temporarily stored. Asearch algorithm was then employed to find the locations of all “seen”points on the perimeter of the “unseen” points' locations. Those twosets of points were then retested, to eliminate random response errorsby the participant, and ensure continuity of the visual field regions.False positive responses, false negative responses and fixation losses(if any) were calculated and reported by the end of the test.Consequently, all the 52 responses were interpolated using a cubicmethod to generate a continuous visual field plot of the testedparticipant.

The visual field test was tried on 20 volunteer subjects using simulatedfield defects, by covering parts of the inner display lens of thespectacles device. The results were assessed on point by pointcomparison basis with an image showing the covered areas of the display.The 52 responses were compared at the approximate correspondinglocations in the covered headset's display image, as a measure oftesting accuracy. Summary of the calculated errors are listed in Table1.

TABLE 1 Error calculations for the 20 cases simulated defects visualfield measurements. Left Eyes Right Eyes Total Error Mean SD Mean SDMean SD Error Points 1.600 1.698 1.500 1.396 1.550 1.535 Error 3.137%3.329% 2.941% 2.736% 3.039% 3.009% Percentage

On the other hand, visual field tests for the 23 clinical patients werecompared with the most recent Humphrey Field Analyzer (HFA) testroutinely made by the subject during their visits. The common 24 degreescentral areas were matched and compared between the two field testingdevices. The comparison and relative error calculations were based againon a point by point basis at the common central 24 degrees areas, whereareas beyond this region were judged through continuity with the centralarea and lack of isolated response points. Summary of the calculatederrors are listed in table 2.

TABLE 2 Error calculations for 23 patients visual field measurements.Left Eyes Right Eyes Total Error Mean SD Mean SD Mean SD Error Points3.059 2.277 3.063 2.061 3.061 2.120 Error 7.647% 5.692% 7.656% 5.039 %7.652% 5.301% Percentage

An image remapping process was then performed, which involved findingnew dimensions and a new center for the displayed images to be shown tothe patient. The output image fits in the bright visual field of asubject's eye by resizing and shifting the original input image.

The visual field was binarized by setting all seen patient responses toones, and keeping the unseen responses to zeros, this resulted in asmall binary image of 8×8 size. In other embodiments, smaller or largerbinary images sizes may be used. Small regions containing at most 4connected pixels, were removed from the binary visual field image. The 4connected pixels represented a predetermined threshold value fordetermination of small regions, although larger or smaller thresholdvalues may be used in some embodiments. Those small regions were notconsidered in the image fitting process. The ignored small regionsrepresent either the normal blind spots, insignificant defects, or anyrandom erroneous responses that might have occurred during the subject'svisual field test.

Based on this interpolated binary field image, the bright field's regionproperties were calculated. Calculated properties for the bright regionsincluded: 1) bright areas in units of pixels, 2) regions' bounding box,3) weighted area centroid, and 4) a list of all pixels constituting thebright regions of the visual field. A bounding box was taken as thesmallest rectangle enclosing all pixels constituting the bright region.A region's centroid was calculated as the center of mass of that regioncalculated in terms of horizontal and vertical coordinates. The valuesof this property correspond to the output image's new center, whichcorresponds to an amount of image shift required for mapping.

Using a list of pixels constituting the largest bright field, the widthsand heights of all pixels bounding the bright field were calculated, asshown in FIG. 37. For each row in the bright field, the two boundingpixels were found, and their vertical coordinates were subtracted to getthe field's width BF_(widths) at that specific row. This widthcalculation was iterated for all rows establishing the considered brightfield to calculate BF_(widths). The same iteration process may beapplied on a column basis to calculate BF_(heights). Afterwards, eitherone of two scaling equations may be used to determine the new size ofthe mapped output image; Width_(map) and Height_(map), as shown in FIG.37.

The Width_(map) may be calculated using resizing equation:

${{Width}_{{map}1} = \frac{{median}\left( {BF}_{widths} \right)}{50}},{{Height}_{{map}1} = {{median}\left( {BF}_{hegiths} \right)}},$

where BF_(widths) and BF_(heights) are the calculated bright field'sbounding pixels' widths and heights, respectively. This scaling methodcalculates the new output image size as the median of the bright visualfield size in each direction, centered at the new image center, found asabove. The median measure was used rather than the mean value, to avoidany resizing skewness related to exceedingly large or small bright fielddimensions. The mapping behavior of this method is to fit images withinthe largest possible bright area, but image stretching or squeezingcould occur, as this method does not preserve the aspect ratio.

The Height_(map) may be calculated using resizing equation:

${{Width}_{{map}2} = {\frac{\sum_{{BF}_{widths}}}{{I{size}}^{2}} \times \;{BX}_{Width}}},{{Height}_{{map}2} = {\frac{\sum_{{BF}_{heights}}}{{I{size}}^{2}} \times {BX}_{height}}},$

where I_(size) is the interpolated image size (output image size),BX_(widths), BX_(heights) are the bounding box width and height. Thesummations in the numerators of the equation approximate the brightfield area calculated with respect to the horizontal and verticaldirections, respectively. Therefore, dividing those summations by thesquare of the output image's size provided an estimate of theproportional image areas to be mapped in each direction. Theseproportions are then multiplied by the corresponding bounding boxdimension that was previously calculated. The mapping behavior of thismethod is to fit images in the largest bright visual field while tryingto preserve the output image's aspect ratio. Incorporating the boundingbox's dimensions into the calculations helped this effect to happen.Yet, preservation of the aspect ratio may not result in all defectivevisual field patterns.

In one embodiment, the AI system may utilize the two equations and tensif not hundreds of the different equations in a process of optimizationto see which one will allow fitting more of the seeing visual field withthe image. Based on the feedback of the operators the system may learnto prefer an equation more than the others based on the specific visualfield to be corrected.

These remapping techniques were used in an identifying hazardous objectstest. The remapping methods were tested on 23 subjects using test imagesthat included a safety hazard, a vehicle in this test. The test imageswere chosen to test the four main quadrants of the visual field, asshown in FIG. 38. A visual field example was used to remap the testimages for display to the subject. The subject was tested by showing animage of an incoming car. The subject could not see the car before beingshown the remapped image, as shown in FIG. 39A illustrating the image asseen by the subject without remapping and in FIG. 39B illustrating theimage as seen after remapping. Our preliminary study demonstrated that78% subjects (18 out of 23) were able to identify safety hazards thatthey could not do without our aid. Some subjects were tested on botheyes individually, so 33 eye tests were available. It was found that in23 out of 33 eyes the visual aid was effective in helping the subjectidentify the simulated incoming hazard (P=0.023).

As indicated, in some embodiments, with respect to FIG. 1A, testingsubsystem 122 may determine one or more defective visual field portionsof a visual field of a user based on a response of the user's eyes to aset of stimuli displayed to the user or lack of response of the user'seyes to the set of stimuli (e.g., eye movement response, pupil sizeresponse, etc.). In some embodiments, one or more moving stimuli may bedynamically displayed to the user as part of a visual test presentation,and the responses or lack of responses to a stimulus may be recorded andused to determine which part of the user's visual field is intact. As anexample, in a kinetic part of the visual test presentation, recording ofresponses of a patient's eyes may begin after a stimulus is displayed inthe visual test presentation and continues until the stimulus disappears(e.g., the stimulus may move from a starting point to a center point ofthe visual test presentation and then disappear). As another example,during the visual test presentation, the stimulus may be removed (e.g.,disappear from the patient's view) when it is determined that thepatient recognizes it (e.g., the patient's gaze direction changes to thecurrent location of the stimulus). As such, the duration of the visualtest presentation may be reduced and more interactive (e.g., the patientis provided with a feeling of playing a game rather than diagnosis ofvisual defects). Based on the foregoing indications (of responses orlack thereof to the set of stimuli), testing subsystem 122 mayautomatically determine the defective visual field portions of theuser's visual field.

In some embodiments, testing subsystem 122 may determine one or moredefective visual field portions of a visual field of a user, andvisioning subsystem 124 may provide an enhanced image or cause anadjustment of one or more configurations of a wearable device based onthe determination of the defective visual field portions. As an example,the enhanced image may be generated or displayed to the user such thatone or more given portions of the enhanced image (e.g., a region of theenhanced image that corresponds to a macular region of the visual fieldof an eye of the user or to a region within the macular region of theeye) are outside of the defective visual field portion. As anotherexample, a position, shape, or size of one or more display portions ofthe wearable device, a brightness, contrast, saturation, or sharpnesslevel of such display portions, a transparency of such display portions,or other configuration of the wearable device may be adjusted based onthe determined defective visual field portions.

FIG. 4 illustrates a process 400 illustrating an example implementationof both a testing mode and a subsequent visioning mode. At a block 402,in a testing mode, data is obtained from diagnostic devices like imagesensors embedded within spectacles device and other user input devices,such as a cellular phone or tablet PC. At a block 404, testing modediagnostics may be performed to detect and measure ocular anomalies fromthe received data (e.g., visual field defects, eye misalignment, pupilmovement and size, images of patterns reflected from the surface of thecornea or the retina, etc.). In an example, a control program andalgorithms were implemented using MATLAB R2017b (MathWorks, Inc.,Natick, Mass., USA). In various embodiments, a subject or tester may beprovided with an option to select to test each eye individually, or testboth eyes sequentially in one run. In some embodiments, the testing modemay include an applied fast thresholding strategy including contraststaircase stimuli covering central radius of 20 degrees or more usingstimuli sequences at predetermined locations. As an example, the testingmode may include an applied fast thresholding strategy include fourcontrast staircase stimuli covering the central 40 degrees' radius using52 stimuli sequences at predetermined locations, as discussed hereinregarding FIGS. 35J and 36. As another example, the testing mode mayinclude the automated determination of the visual defects (e.g.,defective virtual field portions) based on one or more responses of theuser's eyes to a set of stimuli displayed to the user or lack of suchresponses of the user's eyes to the set of stimuli (e.g., eye movementresponse, pupil size response, etc.) as described herein.

At a block 406, the determined diagnostic data may be compared to adatabase or dataset that stores modification profiles for compensatingfor identifiable ocular pathologies (e.g., FIG. 16 and relateddiscussions).

The identified modification profiles may then be personalized to theindividual, for example, to compensate for differences in visual axis,visual field defects, light sensitivity, double vision, change in thesize of the image between the two eyes, image distortions, decreasedvision.

The personalized profiles may be used by a block 408, along withreal-time data to process the images (e.g., using an image processor,scene processing module, and/or visioning module). The real-time datamay include data detected by one or more inward directed image sensors410, providing pupil tracking data, and/or from one or more outwarddirected image sensors comprising one or more visual field cameras 412positioned to capture a visual field screen. At a block 414, real-timeimage correction may be performed and the images may be displayed (block416) on the spectacles device, either as displayed recreated digitalimages, as augmented reality images passing through the spectaclesdevice with corrected portions overlaid, or as images projected into theretinas of the subject. In some examples, the operation of block 414 isperformed in combination with a calibration mode 418 in which the usercan tune the image correction using a user interface such as an inputdevice that allows a user to control image and modification profiles.For example, users can displace the image of one eye to the side, up anddown or cyclotorted to alleviate double of vision. In the above oranother example, a user may fine tune the degree of visual fieldtransformation (e.g., fish eye, polynomial, or conformal) or translationto allow enlarging the field of vision without negatively impact thefunctional vision or cause unacceptable distortions, fine tune thebrightness, and contrast, or invert colors.

FIG. 5 illustrates another example process 500, similar to that ofprocess 400, for implementation of a testing mode and visioning mode. Ata block 502, data for high and low order aberrations for pupil size,degree of accommodation, and gaze, are collected. In some embodiments,all or a portion of the data may be collected from an aberrometer or bycapturing the image of a pattern or grid projected on the cornea and/orretina and comparing it to the reference image to detect aberrations ofthe cornea or the total ocular optical system. The collected data may besent to a vision correction framework, which, at a block 504, maydetermine personalized modification profiles similar to block 406described above. Blocks 508-518 perform similar functions tocorresponding blocks 408-418 in process 400.

FIG. 8 illustrates a workflow 800 showing a testing module 802 thatgenerates and presents a plurality of visual stimuli 804 to a user 806through the spectacles device. The user 806 has a user device 808through which the user may interact to provide input response to thetesting stimuli. In some examples, the user device 808 may comprise ajoystick, electronic clicker, keyboard, mouse, gesture detector/motionsensor, computer, phone such as a smart phone, dedicated device, and/ora tablet PC through which that the user may interfaces to provide inputresponse to the testing stimuli. The user device 808 may also include aprocessor and memory storing instructions that when executed by theprocessor generate display of a GUI for interaction by the user. Theuser device 808 may include a memory, a transceiver (XVR) fortransmitting and receiving signals, and input/output interface forconnecting wired or wirelessly with to a vision correction framework810, which may be stored on an image processing device. The visioncorrection framework 810 may be stored on the spectacles device, on theuser device, etc.—although in the illustrated example, the framework 810is stored on an external image processing device. The framework 810receives testing mode information from the testing module 802 and userinput data from the user device 808.

FIG. 9 illustrates a testing mode process 900, as may be performed bythe workflow 800. At a block 902, a subject is provided a plurality oftesting stimuli according to a testing mode protocol. That stimuli mayinclude images of text, images of objects, flashes of light, patternssuch as grid patterns. The stimuli may be displayed to the subject orprojected onto the retina and/or cornea of the subject. At a block 904,a vision correction framework may receive detected data from one or moreinward directed image sensors, such as data corresponding to a pupilphysical condition (e.g., visual axis, pupil size, and/or limbus). Theblock 904 may further include receiving user response data collectedfrom the user in response to the stimuli. At a block 906, the pupilposition condition may be determined across different stimuli, forexample, by measuring position differences and misalignment differencesbetween different stimuli.

At a block 908, astigmatism determinations may be made throughout thefield of vision, which may include analysis of pupil misalignment dataand/or eye aberrations (e.g., projecting references images on the retinaand cornea and comparing the reflected images from the retinal orcorneal surfaces to reference images). At a block 910, total eyeaberrations may be determined (e.g., by projecting reference images ontothe retina and/or cornea and then comparing the reflected images fromthe retinal or corneal surfaces to reference images, such as describedin FIGS. 31A, 32-34 and accompanying discussion). At a block 912, visualdistortions, such as optical distortions such as coma, astigmatism, orspherical aberrations or visual distortions from retinal diseases, maybe measured throughout the field of vision. At a block 914, the visualfield sensitivity may be measured throughout the field of vision. Invarious embodiments of the process of FIG. 9, one or more of blocks904-914 may be optional.

In some examples, the vision systems herein can assess the data from thetesting mode and determine the type of ocular anomaly and the type ofcorrection needed. For example, FIG. 10 illustrates a process 1000comprising an artificial intelligence corrective algorithm mode that maybe implemented as part of the testing mode. A machine learning frameworkis loaded at a block 1002, example frameworks may include,dimensionality reduction, ensemble learning, meta learning,reinforcement learning, supervised learning, Bayesian, decision treealgorithms, linear classifiers, unsupervised learning, artificial neuralnetworks, association rule learning, hierarchical clustering, clusteranalysis, deep learning, semi-supervised learning, etc.

At a block 1004, a visual field defect type is determined. Three examplefield defects are illustrated: uncompensated blind field 1006, apartially blind spot 1008 with lower sensitivity, and an intact visualfield 1010. The block 1004 determines the visual field defect and thenapplies the appropriate correction protocol for the visioning mode. Forexample, for the uncompensated blind field 1006, at a block 1012, avision correction framework tracks vision, such as through pupiltracking using inward directed image sensors and does video tracking ofa moving object in the visual field (e.g., through outward directedimage sensors such as external cameras). In the illustrated example, ata block 1014, safety hazards in regions of blind spots or that aremoving into the regions of blind spots are detected by, for example,comparing the position of the safety hazard to a mapped visual fieldwith defects as measured in the testing mode. At a block 1016, an objectof interest may be monitored at various locations including a centrallocation and a peripheral location.

In the example of a partially blind spot 1008, an augmented visionvisioning mode may be entered at a block 1018, from which an object inthe visual field is monitored by tracking a central portion of thevisual field. At a block 1020, an image segmentation algorithm may beemployed to separate the object from the visual field. An augmentedoutline may also be applied to the object and displayed to the userwherein the outline coincides with identified edges of the segmentedobject. With respect to the intact visual field 1010, at a block 1022, acustomized corrective algorithm may be applied to correct aberrations,visual field detects, crossed eyes, and/or visual distortion.

In some embodiments, testing subsystem 122 may determine multiplemodification profiles associated with a user (e.g., during a visual testpresentation, while an enhanced presentation of live image data is beingdisplayed to the user, etc.). In some embodiments, each modificationprofile may include a set of modification parameters or functions to beapplied to live image data for a given context. As an example, the usermay have a modification profile for each set of eye characteristics(e.g., a range of gaze directions, pupil sizes, limbus positions, orother characteristics). As further example, the user may additionally oralternatively have a modification profile for each set of environmentalcharacteristics (e.g., a range of brightness levels of the environment,temperatures of the environment, or other characteristics).

Based on the eye-related or environment-related characteristicscurrently detected, the corresponding set of modification parameters orfunctions may be obtained and used to generate the enhanced presentationof the live image data. As an example, the corresponding set ofmodification parameters or functions may be obtained (e.g., to beapplied to an image to modify the image for the user) based on thecurrently-detected eye-related characteristics matching a set ofeye-related characteristics associated with the obtained set ofmodification parameters or functions (e.g., the currently-detectedeye-related characteristics fall within the associated set ofeye-related characteristics). In some embodiments, the set ofmodification parameters or functions may be generated based on thecurrently-detected eye characteristics or environmental characteristics(e.g., ad-hoc generation of modification parameters, adjustment of a setof modification parameters or functions of a currently-storedmodification profile associated with the user for the given context,etc.).

In one use case, a wearable device (implementing the foregoingoperations) may automatically adjust brightness of the enhancedpresentation of the live image data for one or more eyes of the userbased on the respective pupil sizes (e.g., where such adjustment isindependent of the brightness of the surrounding environment). As anexample, subjects with anisocoria have unequal pupil size, and thosesubjects have light sensitivity from a single eye, which cannot toleratethe light brightness tolerated by the healthy eye. In this way, thewearable device enables automatic adjustment of brightness for each eyeseparately (e.g., based on the detected pupil size of the respectiveeye).

In another use case, the wearable device may detect pupil size, visualaxis, optical axis, limbus position, line of sight, or other eyeaccommodation state (e.g., including changes to the foregoing) and maychange a modification profile based on the detected states. As anexample, for subjects with higher order aberrations (e.g., errors ofrefraction that are not correctable by spectacles nor contact lenses),the subject's aberrations are dynamic and change according to the pupilsize and the accommodation state of the eye. The wearable device maydetect the state of accommodation by detecting the signs of the nearreflex (e.g., miosis (decrease the size of the pupil) and convergence(inward crossing of the pupil)). Additionally, or alternatively, thewearable device may include a pupil and line of sight tracker to detectthe direction of gaze. As another example, aberrations of the eye changeaccording to the size and position of the aperture of the optical systemand can be measured in relation to different pupil sizes and positionsof the pupil and visual axis. The wearable device may, for example,measure the irregularities on the cornea to determine the higher orderaberrations (e.g., based on the measurements) and calculate themodification profile to address the higher order aberrations. Fordifferent sizes and positions of the pupil and visual axis (or other eyeaccommodation states), different modification profiles may be createdand stored for future use to provide real-time enhancements. One or moreof these detected inputs enable the wearable device to use theappropriate modification profile (e.g., set of modification parametersor functions) to provide enhancements for the user.

As another example, the wearable device may be used to correct forpresbyopia by automatically performing autofocus of the images displayedto the user to provide near vision. To further augment and enhance nearvision, the wearable device may detect where the user is trying to lookat a near target (e.g., by detecting the signs of the near reflex, suchas miosis (decrease in pupil size and convergence (inward movement ofthe eye)) and perform autofocusing for a region of an imagecorresponding to the target that the user is looking (e.g., the portionof the display that the user is looking, the proximate area around anobject at which the user is looking, etc.). Additionally, oralternatively, the wearable device may determine how far the target is(e.g., a target object or area) by quantifying the amount of the nearreflex exerted by the subject and distance of the target from the eye(e.g., via sensors of the wearable device) and provide the adequatecorrection based on the quantified amount and target distance.

As another example, the wearable device may be used to correct fordouble vision (e.g., related to strabismus or other conditions). Thewearable device may monitor the user's eyes and track the user's pupilsto measure the angle of deviation to displace the images projected foreach eye (e.g., in conjunction with detecting strabismus or otherconditions). Because double vision is typically dynamic (e.g., thedouble vision increases or decreases towards one or more gazes), thewearable device may provide the appropriate correction by monitoring theuser's pupils and the user's line of sight. For example, if the user hasan issue in moving the user's right pupil away from the user's nose(e.g., toward to edge of the user's face), then the user's double visionmay increase when the user is looking to the right and may decrease whenthe user is looking to the left. As such, the wearable device maydisplay an enhanced presentation of live image data to each eye suchthat a first version of the enhanced presentation displayed to one ofthe user's eyes reflects a displacement from a second version of theenhanced presentation displayed to the user's other eye (e.g., where theamount of displacement is based on the pupil position and gazedirection) to dynamically compensate for the user's condition (e.g.,strabismus or other condition) and, thus, prevent double vision for allpotential gaze directions.

Although prisms can be applied to shift image in front of the crossedeye (e.g., caused by strabismus or other condition) to correct fordouble vision, prisms are unable to produce torsion of the image and,thus, not useful in correcting for double vision resulting fromconditions that cause images to appear tilted or cyclotorted (e.g.,cyclotropia is a form of strabismus which causes images received fromboth eyes to appear tilted or cyclotorted). In some use cases, thewearable device may monitor the user's eyes to measure the degree ofstrabismus (e.g., including cyclotorsion) by detecting the pupil,limbus, line of sight, or visual axis of both eyes in relation to eachother. Additionally, or alternatively, the wearable device may performsuch measurements by obtaining images of retinas of both eyes andcomparing the structures of the retina and nerve in relation to eachother. In doing so, the wearable device may detect and measure therelative location of those eye structures and any torsion displacement.Such measurements may be provided to a prediction model to predictmodification parameters for the live image processing to correct for thedefect and alleviate the double vision. Continuous feedback may beobtained from sensors of the wearable device (e.g., pupil tracker, gazetracker, tracker based on retina image, etc.) may be used to change themodification profile applied to live image data in real-time. In furtheruse cases, the user may also fine tune the correction. As an example, animage may be displayed to the user on a user interface, and the user maymove the image (or an object represented by the image) (e.g., using ajoystick or other input device) until that image cross in front of oneeye and rotate the object until the object overlaps with the image seenby the other eye. In some embodiments, upon detection of an indicationof double vision, and without any user input explicitly indicating thatthe image should be moved or the amount or position of the movement, thewearable device may automatically move the image that is crossed infront of one eye (e.g., translate or rotate the image) to performmeasurements or corrections related to the double vision.

As with other forms of strabismus, the resulting displacement caused bycyclotropia changes in real-time based on the intended direction ofaction of the paralyzed (or partially paralyzed) muscle associated withthe cyclotropia and when such a patient is looking towards one side orthe other. By tracking the eye characteristics, the wearable device candynamically compensate for the user's condition by displaying anenhanced presentation of live image data to each eye such that a firstversion of the enhanced presentation displayed to one of the user's eyesreflects a displacement from a second version of the enhancedpresentation displayed to the user's other eye (e.g., where the amountof displacement is based on the pupil position and gaze direction).

In some embodiments, with respect to FIG. 1A, upon obtaining feedbackrelated to a set of stimuli (displayed to a user during a visual testpresentation), feedback related to one or more eyes of the user,feedback related to an environment of the user, or other feedback,testing subsystem 122 may provide the feedback to a prediction model,and the prediction model may be configured based on the feedback. Insome embodiments, testing subsystem 122 may obtain a second set ofstimuli (e.g., during the visual test presentation). As an example, thesecond set of stimuli may be generated based on the prediction model'sprocessing of the set of stimuli and the feedback related to the set ofstimuli. The second set of stimuli may be additional stimuli derivedfrom the feedback to further test one or more other aspects of theuser's visual field (e.g., to facilitate more granular correction orother enhancements for the user's visual field). In one use case,testing subsystem 122 may cause the second set of stimuli to bedisplayed to the user (e.g., during the same visual presentation), and,in response, obtain further feedback related to the second set ofstimuli (e.g., the further feedback indicating whether or how the usersees one or more stimuli of the second set). Testing subsystem 122 maythen providing the further feedback related to the second set of stimulito the prediction model, and the prediction model may be furtherconfigured based on the further feedback (e.g., during the visual testpresentation). As an example, the prediction model may be automaticallyconfigured for the user based on (i) an indication of a response of theuser to one or more stimuli (e.g., of the set of stimuli, the second setof stimuli, or other set of stimuli), (ii) an indication of a lack ofresponse of the user to such stimuli, (iii) an eye image captured duringthe visual test presentation, or other feedback (e.g., the predictionmodel may be personalized toward the user based on the feedback from thevisual test presentation). In one use case, for example, the feedbackindicates one or more visual defects of the user, and the predictionmodel may be automatically configured based on the feedback to addressthe visual defects. As another example, the prediction model may betrained based on such feedback and other feedback from other users toimprove accuracy of results provided by the prediction model (e.g.,trained to provide modification profiles described herein, trained togenerate an enhanced presentation of live image data, etc.).

In some embodiments, visioning subsystem 124 may provide live image dataor other data (e.g., monitored eye-related characteristics) to theprediction model to obtain an enhanced image (derived from the liveimage data) and cause an enhanced image to be displayed. In someembodiments, the prediction model may continue to be configured duringthe display of the enhanced image (derived from the live image data)based on further feedback continuously provided to the prediction model(e.g., on a periodic basis, in accordance with a schedule, or based onother automated triggers). As an example, a wearable device may obtain alive video stream from one or more cameras of the wearable device andcause the enhanced image to be displayed on one or more displays of thewearable device (e.g., within less than a millisecond, less than acentisecond, less than a decisecond, less than a second, etc., of thelive video stream being captured by the cameras of the wearable device).In some embodiments, the wearable device may obtain the enhanced imagefrom the prediction model (e.g., in response to providing the live imagedata, monitored eye-related characteristics, or other data to theprediction model). In some embodiments, the wearable device may obtainmodification parameters or functions from the prediction model (e.g., inresponse to providing the live image data, monitored eye-relatedcharacteristics, or other data to the prediction model). The wearabledevice may use the modification parameters or functions to generate theenhanced image from the live image data (e.g., parameters of functionsused to transform or modify the live image data into the enhancedimage). As a further example, the modification parameters may includeone or more transformation parameters, brightness parameters, contrastparameters, saturation parameters, sharpness parameters, or otherparameters.

In an example, a vision correction framework having a machine learningframework with an AI algorithm may be used to create automaticpersonalized modification profiles by applying transformation,translation, and resizing of the field of view to better fit it to theremaining functional visual field. The machine learning framework mayinclude one or more of data collection, visual field classification,and/or regression models. To facilitate recording of participantresponses, quantitative scores, and feedback, a graphical user interface(GUI) and data collection program may be used.

With respect to transformations applied to images in the visioning mode,example transformations of the machine learning framework may includeone or more of: 1) conformal mapping, 2) fisheye, 3) custom 4th orderpolynomial transformation, 4) polar polynomial transformation (usingpolar coordinates), or 5) rectangular polynomial transformation (usingrectangular coordinates) (e.g., FIG. 13).

With respect to translations applied to images in the visioning mode,examples may include one or more of the following. For the centerdetection, weighted averaged of the best center and the closest point tothe center may be used. For example, the closest point may be determinedby finding the nearest point to the center location. The best center maybe determined by one or more of the following: 1) the centroid of thelargest component, 2) the center of the largest inscribed circle,square, rhombus, and/or rectangle, or 3) the center of the local largestinscribed circle, square, rhombus, and/or rectangle (e.g., FIG. 14). Forexample, the framework may search for the largest shape butalliteratively to avoid getting far from the macular vision region, theframework may substitute this by the weighted average of the closestpoint with the methods.

In various embodiments, the AI algorithm may be initially trained usingsimulated visual field defects. For example, to train the AI algorithm,a dataset of visual field defects may be collected. For example, in oneexperimental protocol, a dataset of 400 visual field defects wereobtained from patients with glaucoma. The dataset may be used to createsimulated visual field defects on virtual reality glasses forpresentation to normal subjects for grading. The resulting feedbackobtained from the grading may then be used to train the algorithm.

For example, an AI algorithm that automatically fits an input image toareas corresponding to the intact visual field pattern for each patientindividually may be used. In various embodiments, the algorithm mayinclude at least three degrees of freedom to remap the images, althoughmore or less degrees of freedom may be used. In one example, the degreesof freedom include transformation, shifting, and resizing. The addedimage transformation may preserve the quality of the central area of theimage corresponding to the central vision, where acuity is highest,while condensing the peripheral areas with an adequate amount of imagequality in the periphery. This may be applied such that the producedoverall image content would be noticeable to the patient.

The image transformations included in the AI algorithm may include oneor more of conformal, polynomial or fish eye transformations. In someembodiments, other transformations may be used. The machine learningtechniques may be trained on a labeled dataset prior to performing theiractual task. In one example, the AI algorithm may be trained on a visualfield dataset that incorporates different types of peripheral defects.For example, in one experiment, the dataset included 400 visual fielddefect patterns. The training phase was then guided by normalparticipants to quantitatively score the remapped images generated bythe AI algorithm.

FIG. 11 shows an image 1100 of a test image (stimuli) according to oneexample. The test image 1100 may be designed to measure the acuity, theparacentral vision and/or the peripheral vision. The illustrated testimage displays five letters at the central region, four internaldiamonds 1102 at the paracentral region, and eight external diamonds1104 at the peripheral region as shown in FIG. 11.

To be able to train the AI system, a volume of data is needed, asintroduced above. As an initial step, defective binocular visual fieldsmay be used to simulate binocular vision of patients as shown in FIG.12. Next, the simulated vision may be presented to subjects through thespectacles device. In this way, the input image can be manipulated usingdifferent image manipulations then presented again to the subject tograde the modified vision. The corrected image may be further correctedand presented to the subject in a continually corrective process untilan optimized corrected image is determined. FIG. 13 illustrates examplesof different correction transformations that may be applied to the imageand presented to the user. FIG. 14 illustrates an example of differenttranslation methods (shifting the image to fit it in the intact visualfield). The intact visual field is white and blind visual field isblack.

The AI system may be designed using machine learning models such asartificial neural networks and Support Vector Machines (SVM). In someexamples, the AI system is designed to produce an output comprising anestimate of the best image manipulation methods (e.g., geometrictransformation and translation) through an optimization AI system. Thevision system, in a visioning mode, may presents images manipulatedaccording to the output image manipulation methods to the patientthrough a headset such that the patient experiences the best possiblevision based on his defective visual field. The machine learningframework (also termed herein “AI System”) of the vision correctionframework may trained using the collected data, (e.g., as describedherein). A block diagram of an example AI system 1500 is shown in FIG.15.

A process 1600 of the AI system 1500 is shown in FIG. 16. The input tothe system 1500 includes a test image and a visual field image. The AIsystem 1500 estimates the best geometric transform for the test imagesuch that more details can be presented through the visual field. Then,AI system 1500 estimates the best translation for the test image suchthat the displayed image covers major parts of the visual field. Then,the test image is transformed and translated as shown in FIG. 17. andFIG. 18, respectively. Finally, the image is combined with the visualfield again in case of the training only for the simulation purpose, butit is displayed directly to the patient in the testing phase. Ascreenshot of graphical user interface presenting a summary of visualfield analysis, which may include a final implementation of the visualfield AI system including parameters of the image transformation andtranslation to be applied to the image, is shown in FIG. 19.

In example an implementation, an artificial neural network model wasused to implement the machine learning framework (“AI system”) on thevision correction framework. The AI system takes as the visual fieldimage converted to a vector. The AI system gives as output theprediction of the parameters of the image transformation and thetranslation to be applied to the scene image. Then, the scene image ismanipulated using these parameters. The AI system includes two hiddenlayers wherein each hidden layer includes three neurons (i.e., units)and one output layer. One such example AI system model is shown FIG. 20.This AI system may also extend to convolutional neural network model foreven more accurate results, in other examples. FIGS. 21 and 22illustrate example processes 2100 and 2200 of a testing mode applicationof a neural network and an AI algorithm optimization process using aneural network, respectively.

In some embodiments, with respect to FIG. 1A, upon obtaining feedbackrelated to a set of stimuli (displayed to a user during a visual testpresentation), feedback related to one or more eyes of the user,feedback related to an environment of the user, or other feedback,testing subsystem 122 may provide the feedback to a prediction model,and the prediction model may be configured based on the feedback. Insome embodiments, further feedback may be continuously obtained andprovided to the prediction model (e.g., on a periodic basis, inaccordance with a schedule, or based on other automated triggers) toupdate the configuration of the prediction model. As an example, theconfiguration of the prediction model may be updated while one or moreenhancements of live image data are being displayed to the user.

In some embodiments, visioning subsystem 124 may monitor characteristicsrelated to one or more eyes of the user (e.g., gaze direction, pupilsize or reaction, limbus position, visual axis, optical axis, eyelidposition or movement, head movement, or other characteristics) andprovide the eye characteristic information to the prediction modelduring an enhanced presentation of live image data to the user.Additionally, or alternatively, visioning subsystem 124 may monitorcharacteristics related to an environment of the user (e.g., brightnesslevel of the environment, temperature of the environment, or othercharacteristics). As an example, based on the eye or environmentalcharacteristic information (e.g., indicating the monitoredcharacteristics), the prediction model may provide one or moremodification parameters or functions to be applied to the live imagedata to generate the enhanced presentation of the live image data (e.g.,the presentation of one or more enhanced images derived from the liveimage data to the user). In one use case, the prediction model mayobtain the modification parameters or functions (e.g., stored in memoryor at one or more databases) based on the currently-detected eyecharacteristics or environmental characteristics. In another use case,the prediction model may generate the modification parameters orfunctions based on the currently-detected eye characteristics orenvironmental characteristics.

In some embodiments, with respect to FIG. 1A, visioning subsystem 124may facilitate enhancement of a field of view of a user via one or moredynamic display portions on one or more transparent displays (e.g.,based on feedback related to a set of stimuli displayed to a user orother feedback). As an example, the dynamic display portions may includeone or more transparent display portions and one or more other displayportions (e.g., of a wearable device or other device). In someembodiments, visioning subsystem 124 may cause one or more images to bedisplayed on the other display portions (e.g., such that the images arenot displayed on the transparent display portions). As an example, auser may see through the transparent display portions of a transparentdisplay, but may not be able to see through the other display portionsand instead sees the image presentation on the other display portions(e.g., around or proximate the transparent display portions) of thetransparent display. That is, in some embodiments, a dynamic hybridsee-through/opaque display may be used. In this way, for example, one ormore embodiments can (i) avoid the bulky and heavy weight of typicalvirtual reality headsets, (ii) make use of a user's intact vision (e.g.,making use of the user's good acuity central vision if the user hasintact central vision but a defective peripheral visual field, makinguse of the user's intact peripheral vision if the user has intactperipheral vision but a defective central visual field, etc.), and (iii)mitigate visual confusion that would otherwise be caused by typicalaugmented reality technology that has an overlap effect between thesee-through scene with the internally displayed scene.

As an example, live image data may be obtained via the wearable device,and an enhanced image may be generated based on the live image data anddisplayed on the other display portions of the wearable device (e.g.,display portions of a display of the wearable device that satisfy anopaque threshold or fail to satisfy a transparency threshold). In someembodiments, visioning subsystem 124 may monitor one or more changesrelated to one or more eyes of the user and cause, based on themonitoring, an adjustment of the transparent display portions of thetransparent display. As an example, the monitored changes may include aneye movement, a change in gaze direction, a pupil size change, or otherchanges. One or more positions, shapes, sizes, transparencies,brightness levels, contrast levels, sharpness levels, saturation levels,or other aspects of the transparent display portions or the otherdisplay portions of the wearable device may be automatically adjustedbased on the monitored changes.

In one use case, with respect to FIG. 24A, a wearable device 2400 mayinclude a transparent display 2402 dynamically configured to have atransparent peripheral portion 2404 and an opaque central portion 2406such that the light from the user's environment can directly passthrough the transparent peripheral portion 2404, but does not passthrough the opaque central portion 2406. For patients with diagnosedcentral visual field anomalies 2306, the foregoing dynamic configurationenables such patients to use their intact peripheral visual field toview the actual un-corrected view of the environment and be presentedwith a corrected rendition of the central region on the opaque centralportion 2406.

In another use case, with respect to FIG. 24B, the wearable device 2400may include the transparent display 2402 dynamically configured to havean opaque peripheral portion 2414 and a transparent central portion 2416such that the light from the user's environment can directly passthrough the transparent central portion 2416, but does not pass throughthe opaque peripheral portion 2414. For patients with peripheral visualfield anomalies, the foregoing dynamic configuration enables suchpatients to use their intact central visual field to view the actualun-corrected view of the environment and be presented with a correctedrendition of the peripheral region on the opaque peripheral portion2414. In each of the foregoing use cases, with respect to FIGS. 24A and24B, one or more positions, shapes, sizes, transparencies, or otheraspects of the transparent display portions 2404, 2416 or the opaquedisplay portions 2406, 2414 may be automatically adjusted based onchanges related to one or more eyes of the user that are monitored bythe wearable device 2400 (or other component of system 100).Additionally, or alternatively, one or more brightness levels, contrastlevels, sharpness levels, saturation levels, or other aspects of theopaque display portions 2406, 2414 may be automatically adjusted basedon changes related to one or more eyes of the user that are monitored bythe wearable device 2400. In some cases, for example, to dynamicallyaccommodate for areas of the user's visual field that have reducedbrightness, the user's pupil and line of sight (or other eyecharacteristics) may be monitored and used to adjust the brightnesslevels of parts of the opaque display portions 2406, 2414 (e.g., inaddition to or in lieu of increasing the brightness levels of parts ofthe enhanced image that correspond to the reduced brightness areas ofthe user's visual field).

As an example, with respect to FIG. 24C, based on a determination of auser's visual field (e.g., including defective visual field portions,intact visual field portions, etc., as represented by visual field plane2432), an enhanced image may be generated (e.g., as represented by theremapped image plane 2434) as described herein. The enhanced image maybe displayed to the user on one or more opaque display portions in theopaque area 2438 of a display (e.g., as represented by selectivetransparency screen plane 2416) such that the displayed enhanced imageaugments the user's view of the environment through the transparent area2440 of the display.

In one use case, with respect to FIG. 24C, the selective transparencyscreen plane 2436 may be aligned with the other planes 2432 and 2434 viaone or more eye tracking techniques. As an example, an eye trackingsystem (e.g., of wearable device 2400 or other device) may be calibratedfor a user to ensure proper image projections according to the user'spersonalized intact visual field. The eye tracking system maycontinuously acquire gaze coordinates (e.g., on a periodic basis, inaccordance with a schedule, or other automated triggers). A coordinatestransformation may be performed to convert the eye movements sphericalcoordinates (0, (p) into the display's Cartesian coordinates (x, y). Assuch, the device's controller may determine the central position of theimages to be displayed. Camera images will be truncated and shifted tomatch the acquired gaze vector direction (e.g., FIG. 24C). The sameCartesian coordinates may be sent to the selective transparency screencontroller to make the area corresponding to macular vision at thecurrent gaze direction transparent and allow usage of the central visualacuity. In some cases, low pass filtering may be performed on the gazedata to remove micro-eye movements (e.g., micro-eye movements caused byincessantly moving and drafting that occur even at fixations because theeyes are never completely stationary) that may otherwise cause shakyimages to be displayed to the user.

As indicated above, in some embodiments, the wearable device may beconfigured to selectively control transparency of a display area of amonitor, such as a screen, glass, film, and/or layered medium. FIG. 23illustrates an example process 2300 implementing testing and visioningmodes and the use of a custom-reality spectacles device, which may use amacular (central) versus peripheral vision manipulation.

In some examples, the custom reality spectacles device (e.g., FIGS.40A-40C) include transparent glasses for overlaying corrected imagesonto a visible scene. The glasses may comprise a monitor comprising ascreen having controllably transparency onto which images may beprojected for display. In one example, the display comprises a heads-updisplay. In various embodiments, a custom reality spectacles deviceincludes glasses having controllable layers for overlaying correctedimages onto a scene visible through the glasses. The layers may compriseglass, ceramic, polymer, film, and/or other transparent materialsarranged in a layered configuration. The controllable layers may includeone or more electrically controlled layers that allow for adjusting thetransparency over one or more portions of the visual field, for example,in pixel addressable manner. In one embodiment, may include pixels orcells that may be individually addressable (e.g., via an electriccurrent, field, or light). The controllable layers may be layers thatmay be controlled to adjust contrast of one or more portions of thevisual field, color filtering over portions, the zooming in/zooming outof portions, focal point over portions, transparency of the spectaclesdevice surface that display the image to block or allow the light comingfrom the environment at a specific location of the visual field. Ifthere is a portion of field of view (e.g., a portion of the peripheralvision or a portion of the macular vision or a portion, part of it ismacular and part of it is peripheral) for manipulation to augment asubject's vision, then the transparency of that portion of the glass maybe lowered to block the view of the environment through that portion ofglass and to allow the patient to see more clearly the manipulated imagedisplayed along that portion of the glass. In various embodiments,vision system or custom reality spectacles device may dynamicallycontrol transparency regions to allow a subject to naturally view theenvironment when redirecting eyes by eye movement rather than just headmovement. For example, pupil tracking data (e.g., pupil and/or line ofsight tracking) may be used to modify the portion of the glass havingdecreased transparency such that the decreased transparency regiontranslates relative to the subject's eye.

For example, the transparency of the glass in the spectacles devicecomprising custom-reality glasses may be controllably adjusted to blocklight from that portion of the visual field corresponding to where imagecorrection is performed (e.g., at a central region or a peripheralregion). Otherwise subject may see the manipulated image and see throughit and perceive the underling actual visual field in that region. Suchlight blocking can be achieved by a photochromic glass layer within thespectacles device. Moreover, the spectacles device may change theposition of the area where the glass transparency is reduced bymeasuring for eye (pupil) movement using inward directed image sensors,and compensating based on such movement by processing in the visioncorrection framework. In one example, the display screen of the monitorincludes pixels or cells including electric ink technology and that maybe individually addressed to cause an electric field to modify thearrangement of ink within a cell to modify transparency and/or generatea pixel of the display. In an example implementation, FIG. 40A showscustom-reality glasses 4000 formed for a frame 4002 and two transparentglass assemblies 4004. As shown in FIGS. 40B and 40C, the transparentglass assemblies 4004 have embedded, electronically controllablecorrection layers 4006 that may be controllable from fully transparentto fully opaque, that may be digital layers capable of generating acorrection image to overlay or supplant a portion of the field of viewof the glasses 4000. The correction layers 4006 may be connected,through an electrical connection 4008, to an image processing device4010 on the frame 4002.

With specific reference to the process 2300 of FIG. 23, at a block 2302testing mode data may be received by a vision correction framework, andat a block 2304 visual field distortions, defects, aberrations, and/orother ocular anomalies may be determined, along with their locations.

For diagnosed central visual field anomalies 2306, at a block 2308 thecustom reality spectacles device may allow the image from theenvironment to pass through the glass thereof to a peripheral field ofthe user (e.g., FIG. 24A). As shown, custom reality spectacles device2400 may have a multi-layered glass viewfinder 2402. A peripheral region2404 may be set as transparent to allow light passage there through,allowing the subject to view the actual un-corrected environment. At ablock 2312, a central region 2406 of the environment may be made opaqueby the spectacles device 2400 and a corrected rendition of the centralregion may be presented by display to the user, for example, usingcorrections such as those of FIGS. 13, 14, 17, and 18.

At block 2314, For diagnosed peripheral visual field anomalies(determined at block 2308), a central region 2416 (e.g., FIG. 24B) ofthe environment is allowed to pass through a transparent portion of thespectacles device 2400, and transparency of a peripheral region 2414 ismodified to block such that a corrected peripheral version image may bedisplayed within peripheral region 2414, for example using thecorrective transformations herein.

In some embodiments, with respect to FIG. 1A, visioning subsystem 124may facilitate enhancement of a field of view of a user via projectionsonto selected portions of an eye of the user (e.g., based on feedbackrelated to a set of stimuli displayed to a user or other feedback). Asdiscussed herein, an enhanced presentation of live image data may bedisplayed to the user by projecting the enhanced presentation (e.g.,modified images derived from the live image data) onto the user's eyes.In addition to or alternatively to the use of dynamic display portionson a screen (e.g., to enable the user to see-through one or moreportions of the screen while the user sees modified live image databeing displayed on one or more other portions of the screen), themodified image data may be projected onto one or more portions of an eyeof the user (e.g., one or more portions of a retina of the user) whilesimultaneously avoiding projection of the modified image data onto oneor more other portions of the user's eye (e.g., one or more otherportions of the retina of the user).

In some embodiments, the modified image data may be projected onto oneor more intact visual field portions of an eye of the user whilesimultaneously avoiding projection of the modified image data onto oneor more other intact visual field portions of the user's eye. As anexample, with respect to the other intact visual field portions whereprojection of the modified image data is avoided, light from the user'senvironment can pass through the user's retinas (e.g., without anysignificant interference from light being emitted by the projector),thereby allowing the user to see the environment via such other intactvisual field portions. On the other hand, with respect to the intactvisual field portions onto which the modified image data is beingprojected, the projected light prevents the user from seeing theenvironment via the projected-onto portions of the user's intact visualfield. Nevertheless, by projecting the modified live image data ontothose intact visual field portions of the user's eyes, the system allowsthe modified live image data to be used to augment the user's visualfield (e.g., in a manner similar to the use of dynamic display portionsto augment the user's visual field).

In some embodiments, visioning subsystem 124 may monitor one or morechanges related to one or more eyes of the user and cause, based on themonitoring, an adjustment of one or more projecting portions of aprojector (e.g., portions including laser diodes, LED diodes, etc., thatare emitting a threshold amount of light visible to the user's eyes). Asan example, as with the adjustment of a dynamic display portion on ascreen, the monitored changes may include an eye movement, a change ingaze direction, a pupil size change, or other changes. One or morepositions, shapes, sizes, brightness levels, contrast levels, sharpnesslevels, saturation levels, or other aspects of the projecting portionsor other portions of the projector may be automatically adjusted basedon the monitored changes.

In one use case, a wearable device may include a projector configured toselectively project an enhanced presentation (e.g., modified imagesderived from live image data) onto one or more portions of the user'seyes (e.g., one or more portions of each retina of the user thatcorrespond to the user's intact visual field) while simultaneouslyavoiding projection of the modified image data onto one or more otherportions of the user's eyes (e.g., one or more other portions of eachretina of the user that correspond to the user's intact visual field).In some cases, alignment of such a selective projection plane may bealigned with the other planes (e.g., a visual field plane, a remappedimage plane, etc.) via one or more eye tracking techniques (e.g., one ormore techniques similar to those described in FIG. 24C with respect tothe use of dynamic display portions on a screen).

With respect to FIG. 24A, a wearable device 2400 may include atransparent display 2402 dynamically configured to have a transparentperipheral portion 2404 and an opaque central portion 2406 such that thelight from the user's environment can directly pass through thetransparent peripheral portion 2404, but does not pass through theopaque central portion 2406. For patients with diagnosed central visualfield anomalies 2306, the foregoing dynamic configuration enables suchpatients to use their intact peripheral visual field to view the actualun-corrected view of the environment and be presented with a correctedrendition of the central region on the opaque central portion 2406.

In another use case, with respect to FIG. 24B, the wearable device 2400may include the transparent display 2402 dynamically configured to havean opaque peripheral portion 2414 and a transparent central portion 2416such that the light from the user's environment can directly passthrough the transparent central portion 2416, but does not pass throughthe opaque peripheral portion 2414. For patients with peripheral visualfield anomalies, the foregoing dynamic configuration enables suchpatients to use their intact central visual field to view the actualun-corrected view of the environment and be presented with a correctedrendition of the peripheral region on the opaque peripheral portion2414. In each of the foregoing use cases, with respect to FIGS. 24A and24B, one or more positions, shapes, sizes, transparencies, or otheraspects of the transparent display portions 2404, 2416 or the opaquedisplay portions 2406, 2414 may be automatically adjusted based onchanges related to one or more eyes of the user that are monitored bythe wearable device 2400 (or other component of system 100).Additionally, or alternatively, one or more brightness levels, contrastlevels, sharpness levels, saturation levels, or other aspects of theopaque display portions 2406, 2414 may be automatically adjusted basedon changes related to one or more eyes of the user that are monitored bythe wearable device 2400. In some cases, for example, to dynamicallyaccommodate for areas of the user's visual field that have reducedbrightness, the user's pupil and line of sight (or other eyecharacteristics) may be monitored and used to adjust the brightnesslevels of parts of the opaque display portions 2406, 2414 (e.g., inaddition to or in lieu of increasing the brightness levels of parts ofthe enhanced image that correspond to the reduced brightness areas ofthe user's visual field).

In some embodiments, testing subsystem 122 may monitor one or moreeye-related characteristics related to eyes of a user during visual testpresentation via two or more user interfaces (e.g., on two or moredisplays) and determine visual defect information for one or more eyesof the user based on the eye-related characteristics occurring duringthe visual test presentation. As an example, testing subsystem 122 maycause one or more stimuli to be presented at one or more positions on atleast one of the user interfaces and generate the visual defectinformation for an eye of the user based on one or more eye-relatedcharacteristics of the eye occurring upon the stimuli presentation. Insome embodiments, a deviation measurement for the eye may be determinedbased on the eye-related characteristics (indicated by the monitoring asoccurring upon the stimuli presentation) and used to provide correctionsor other enhancements for the eye. As an example, the deviationmeasurement may indicate a deviation of the eye relative to the othereye, and the deviation measurement may be used to determine and correctfor double vision or other vision defects. As an example, the amount ofmovement indicates the amount of eye crossing (e.g., strabismus), andthe direction (or axis) of the movement indicates the type ofstrabismus. For example, if the eye movement was from “out” to “in,”that means the strabismus is crossing out (e.g., exotropia). As such, insome embodiments, double vision may be autonomously determined andcorrected via a wearable device.

In some embodiments, testing subsystem 122 may determine a deviationmeasurement or other visual defect information for a first eye of a userby (i) causing a stimulus to be presented at a position on a first userinterface for the first eye while a stimuli intensity of a second userinterface for a second eye of the user does not satisfy a stimuliintensity threshold and (ii) determining the visual defect informationbased on one or more eye-related characteristics of the first eyeoccurring upon the stimulus presentation. As an example, the stimuluspresentation on the first user interface may occur while a stimulus isnot presented on the second user interface. In one use case, if thefirst eye (e.g., right eye) is crossed outside immediately prior to suchstimulus presentation on the first user interface (e.g., FIG. 25D), bypresenting the stimulus in front of the first eye only (e.g., right eyeonly), the first eye will instinctively move toward and fixate on thestimulus position (e.g., within less than a second) because the secondeye (e.g., left eye) will lose any dominance it had as a result ofhaving nothing to look at. Testing subsystem 122 may measure thecorrection movement of the first eye (and other changes in theeye-related characteristics of the first eye) to determine the deviationmeasurement for the first eye. As an example, the amount of movement ofthe first eye that occur upon such stimulus presentation may correspondto the amount of the crossing of the first eye.

In some embodiments, testing subsystem 122 may determine a deviationmeasurement or other visual defect information for a first eye of a userby (i) causing a stimulus to be presented at a given time at thecorresponding position on a first user interface for the first eye andat the corresponding position on a second user interface for the secondeye and (ii) determining the visual defect information based on one ormore eye-related characteristics of the first eye occurring upon thestimulus presentation. As an example, the target stimulus may bepresented at the central position on both user interfaces or at anothercorresponding position on both user interfaces. In one use case, whenpresenting a stimulus in front of both eyes (e.g., FIG. 25B), thedominant eye (e.g., the left eye in FIG. 25B) will instinctively move tothe corresponding position and fixate on the target stimulus (e.g.,within less than a second). Although the other eye (e.g., the right eyein FIG. 25B) will also move, it will not instinctively fixate on thetarget stimulus because the other eye is crossed out, thereby causingthe user to see double. For example, while the other eye willinstinctively move, the instinctive movement will result in the othereye's gaze direction being toward a different position. However, whenthe user focuses on looking at the target stimulus with the user's othereye, the other eye will move and fixate on the target stimulus presentedat the corresponding position on the other eye's user interface. Becausethe target stimulus is presented at the corresponding position on bothuser interfaces, the dominant eye will remain dominant and continue tofixate on the target stimulus presented at the corresponding position onthe dominant eye's user interface. Testing subsystem 122 may measure thecorrection movement of the other eye (and other changes in theeye-related characteristics of the other eye) to determine the deviationmeasurement for the other eye (e.g., the amount of movement of the othereye may correspond to the amount of the crossing of the other eye).

In some embodiments, after obtaining a deviation measurement or othervisual defect information for a first eye of a user by measuring changesin the eye-related characteristics of the first eye (e.g., the movementof the first eye occurring upon the presentation of a stimulus at acorresponding position on a first user interface for the first eye),testing subsystem may cause a stimulus to be presented at a modifiedposition on the first user interface for the first eye display. As anexample, the stimulus presentation at the modified position occurs whilea stimulus is not presented on a second user interface for the secondeye (or at least while a stimuli intensity of the second user interfacedoes not satisfy a stimuli intensity threshold so that the second eyedoes not react to any stimuli on the second user interface). Based onone or more eye-related characteristics of the first eye or the secondeye not changing beyond a change threshold upon the presentation at themodified position, testing subsystem 122 may confirm the deviationmeasurement or other visual defect information for the first eye. As anexample, the deviation measurement for the first eye may be confirmedbased on the first eye not moving beyond a movement threshold (e.g., nomovement or other movement threshold) upon the presentation of astimulus at the modified position. Additionally, or alternatively, thedeviation measurement for the first eye may be confirmed based on thesecond eye not moving beyond the movement threshold.

In some embodiments, testing subsystem 122 may generate one or moremodification profiles associated with a user based on one or moredeviation measurements or other visual defect information for one ormore eyes of the user (e.g., that are obtained via one or more visualtest presentations). As an example, each of the modification profilesmay include modification parameters or functions used to generate anenhanced image from live image data (e.g., parameters of functions usedto transform or modify the live image data into the enhanced image). Assuch, in some embodiments, visioning subsystem 124 may generate modifiedvideo stream data to be displayed to the user based on (i) video streamdata representing an environment of the user and (ii) the modificationprofiles associated with the user.

As an example, a visual test may be performed to determine whether adeviation of an eye of a user exists, measure a deviation of an eye ofthe user, or generate one or more modification profiles for an eye ofthe user. In one use case, with respect to FIG. 25A, when the targetstimulus 2502 is presented at the central position on right and leftdisplays 2503 a and 2503 b of a wearable device to a patient (e.g.,patient with no crossed eyes), both eyes (e.g., right and left eyes 2504a and 2504 b) will instinctively move and fixate on the target stimulus2502 at the central position on each wearable display, and, thus, thepatient only sees one target stimulus 2502. As such, based on theforegoing eye responses, testing subsystem 122 may determine that theuser does not have double vision.

In another use case, with respect to FIG. 25B, when the target stimulus2502 is presented at the central position on right and left displays ofa wearable device to a patient with crossed eyes, one of the eyes (e.g.,the dominant eye) will instinctively move to the central position andfixate on the target stimulus 2502 (e.g., the left eye 2504 binstinctively fixated on the target stimulus 2502). Although the othereye (e.g., the right eye 2504 a) will also move, it does not fixate onthe target stimulus 2502 because the other eye is crossed out, therebycausing the user to see double (e.g., the user sees two target stimuliinstead of one). For example, while the other eye will instinctivelymove, the instinctive movement will result in the other eye's gazedirection being toward a different position. Based on the foregoing eyeresponses, testing subsystem 122 may determine that the user has doublevision. However, in a further use case, when the user focuses on lookingat the target stimulus 2502 with the user's other eye (e.g., the crossedright eye 2504 a), the other eye will move and fixate on the targetstimulus 2502 presented at the central position on the other eye's userinterface. Because the target stimulus 2502 is presented at the centralposition on both displays 2503 a and 2503 b, the dominant eye willremain dominant and continue to fixate on the target stimulus 2502presented at the central position on the dominant eye's display. Thecorrection movement of the other eye (and other changes in theeye-related characteristics of the other eye) may be measured todetermine the deviation measurement for the other eye (e.g., the amountof movement of the other eye may correspond to the amount of thecrossing of the other eye).

In another use case, with respect to FIG. 25C, at time t1, a stimulus(e.g., the target stimulus 2502) may be presented at the centralposition only to the left eye 2504 b by presenting the stimulus on theleft display 2503 b and not presenting a stimulus on the right display2503 a. If, for example, a stimulus was presented at the centralposition to both eyes 2504 a and 2504 b as shown in FIG. 25B immediatelyprior to the stimulus presentation to only the left eye 2504 b (e.g., attime t0 immediately prior to the stimulus presentation at time t1), thenthe left eye 2504 b will not move because the left eye 2504 b is alreadyfixated on the central position. If, however, the left eye 2504 b is notalready fixated on the central position, the stimulus presentation toonly the left 2504 b will cause the left eye 2504 b to instinctivelymove to the central position and fixate on the target stimulus 2502.

As indicated in FIG. 25D, a stimulus (e.g., the target stimulus 2502)may be presented at the central position only to the right eye 2504 a(e.g., at time t2) by presenting the stimulus on the right display 2503a and not presenting a stimulus on the left display 2503 b. Because theleft eye 2504 b is not being stimulated (e.g., has nothing to look at),the left eye 2504 b will lose dominance and thus move to the outside asa result of the right eye 2504 a taking over. Upon presenting the targetstimulus 2502 only to the right eye 2504 a, the right eye 2504 a willinstinctively take over and move to fixate on the central position.Testing subsystem 122 may measure the movement of the right eye 2504 ato determine the deviation measurement for the right eye 2504 a (e.g.,the amount of movement may correspond to the amount of the crossing ofthe right eye 2504 a).

As indicated in FIG. 25E, a stimulus (e.g., the target stimulus 2502)may be presented at the central position to both eyes 2504 a and 2504 b(e.g., at time t3) by presenting the stimulus on the left display 2503 band on the right display 2503 a. If crossing is alternating (nodominance of either eye), the right eye 2504 a will stay fixating on thecentral position, and the left eye 2504 b will stay crossed. If,however, the left eye 2504 b is the dominant eye (as indicated in FIG.25E), the left eye 2504 b will instinctively move and fixate on thecentral position. The movement of the left eye 2504 b will cause theright eye 2504 a to be crossed, resulting the right eye 2504 a's gazedirection being toward a different position. Testing subsystem 122 maymeasure the movement of the left eye 2504 b to determine or confirm thedeviation measurement for the right eye 2504 a (e.g., the amount ofmovement of the left eye 2504 b may correspond to the amount ofdeviation of the right eye 2504 a).

In a further use case, further testing may be performed to confirm thedeviation measurement for the non-dominant eye. For example, asindicated in FIG. 25F, subsequent to one or more of the foregoing stepsdescribed with respect to FIGS. 25B-25E, a stimulus (e.g., the targetstimulus 2502) may be presented at the central position only to the lefteye 2504 b (e.g., at time t4) by presenting the stimulus on the leftdisplay 2503 b and not presenting a stimulus on the right display 2503a. To the extent that the left eye 2504 b lost fixation (e.g., due tothe presentation in FIG. 25E), the presentation in FIG. 25F will causethe left eye 2504 b to instinctively move to gain fixation on thecentral position. The movement of the left eye 2504 b will cause theright eye 2504 a to be crossed, resulting in the right eye 2504 a's gazedirection being toward a different position. As indicated in FIG. 25G,based on the deviation measurement for the right eye 2504 a, a modifiedposition may be determined for presenting a stimulus to the right eye2504 a. As such, while the target stimulus 2502 is being presented atthe central position on the left display 2503 b, the target stimulus2502 may be presented at the modified position on the right display 2503a (e.g., at time t5).

Subsequently, with respect to FIG. 25H, the target stimulus 2502 mayonly be presented to the right eye 2504 a (e.g., at time t6) bypresenting the target stimulus 2502 at the modified position on theright display 2503 a and not presenting a stimulus on the left display2503 b. Specifically, for example, the target stimulus 2502 is deviatedto the right by the same amount as the deviation measured in one or moreof the foregoing steps described with respect to FIGS. 25B-25E. If thedeviation measurement is accurate, the right eye 2504 a will not move.If the deviation measurement is not accurate, the right eye 2504 a willslightly move, and the amount of movement may be measured by thewearable device (e.g., the pupil tracker of the wearable device) and themeasurement of the slight movement may be used to fine tune thedeviation. As an example, the measurement and the modified position maybe used to determine an updated modified position for presenting astimulus to the right eye 2504 a, and one or more of the steps describedwith respect to FIGS. 25F-25H may be repeated using the updated modifiedposition. Additionally, or alternatively, one or more of the steps ofFIGS. 25B-25E may be repeated to redetermine the deviation measurementfor one or more eyes of the user (e.g., redetermining the deviationmeasurement for the right eye 2504 a). With respect to FIG. 25I, thetarget stimulus 2502 may then be presented to both eyes 2504 a and 2504b (e.g., at time t7) by presenting the target stimulus 2502 at themodified position on the right display 2503 a and at the centralposition on the left display 2503 b. Because the target stimulus 2502 infront of the right eye 2504 a is deviated to the right in accordancewith the deviation measurement (e.g., as determined or confirmed in oneor more of the foregoing steps), the user is no longer seeing double,thereby providing autonomous correction for the patent's double vision.

In some embodiments, a visual test may be performed to determine whicheye of a user is a deviating eye. Based on such determination, adeviation of the deviating eye may be measured, and the deviationmeasurement may be used to generate a modification profile to correctthe deviation of the user's vision. As an example, testing subsystem 122may cause a stimulus to be presented at a given time at a first positionon a first user interface for a first eye and at the first position on asecond user interface for a second eye. Testing subsystem 122 may detectlack of fixation of the first eye on the first position upon thestimulus presentation of a stimulus on the first user interface. Basedon the detection of the lack of fixation of the first eye, testingsubsystem 122 may determine the first eye of the user to be a deviatingeye. As an example, with respect to FIG. 25B, when the target stimulus2502 is presented at the central position on right and left displays ofa wearable device to a patient with crossed eyes, one of the eyes (e.g.,the dominant eye) will instinctively move to the central position andfixate on the target stimulus 2502 (e.g., the left eye 2504 binstinctively fixated on the target stimulus 2502). Although the othereye (e.g., the right eye 2504 a) will also move, it does not fixate onthe target stimulus 2502 because the other eye is crossed out, therebycausing the user to see double (e.g., the user sees two target stimuliinstead of one). Based on this detected lack of fixation, the other eyemay be determined to be the deviating eye.

In some embodiments, a visual test may be performed while the eye islooking in different directions of gaze to detect how much is the doublevision in each direction of gaze. In this way, diagnostics andcorrection may be performed for the specific type of strabismus (e.g.,incomitant strabismus). For example, patient with paralysis of a muscleof the eye, the deviation between both eyes (angle of strabismus) islarger when the eye is looking towards the direction of action of thatmuscle. For example, if the muscle that takes the left eye out isparalyzed, then the left eye will be looking in (aka esotropia). Theesotropia degree will be more if the left eye is trying to look out.This phenomenon happens with paralytic strabismus. By repeating thequantification test while the stimulus is presented in different areasof the field of vision, the wearable device (or other components inconnection with the wearable device) may accurately measure the angle ofdeviation. Also, knowing the degree of deviation in different directionsof gaze will enable dynamic correction for double vision. When suchvisual test presentation is provided via a wearable device, and when thepupil tracker of the wearable device detects that the eye at a specificgaze, the wearable device may provide the image displacement thatcorresponds to that gaze.

In some embodiments, such tests can be done while patient looking at adistance object and at a near object. In some embodiments, the wearabledevice may automatically test for the range of motion of the extraocularmuscle by presenting a stimulus that moves around. As the patientfollows it with his eyes, the wearable device (or other components inconnection with the wearable device) measures the range of movement anddetermines information regarding the double vision of the user based onthe range of movement measurement.

Thus, in some embodiments, multiple modification profiles may begenerated for a user to correct for dynamic vision defects (e.g., doublevision or other vision defects). As an example, a first modificationprofile associated with the user may include one or more modificationparameters to be applied to modify an image for a first eye of the userin response to the second eye's gaze direction being directed at a firstposition, the second eye having a first torsion (e.g., first angle oftorsion), or other characteristic of the second eye. A secondmodification profile associated with the user may include one or moremodification parameters to be applied to modify an image for the firsteye in response to the second eye's gaze direction being directed at asecond position, the second eye having a second torsion (e.g., secondangle of torsion), or other characteristic of the second eye. A thirdmodification profile associated with the user may include one or moremodification parameters to be applied to modify an image for the firsteye in response to the second eye's gaze direction being directed at athird position, the second eye having a third torsion (e.g., third angleof torsion), or other characteristic of the second eye, and so on. Inone use case, one or more of the steps described with respect to FIGS.25B-25H may be repeated for one or more other positions (in addition oralternatively to the central position) to generate multiple modificationprofiles for the user.

In some embodiments, visioning subsystem 124 may monitor one or moreeye-related characteristics of one or more eyes of the user and maygenerate modified video stream data to be displayed to the user based on(i) video stream data representing an environment of the user, (ii) themonitored eye-related characteristics, and (iii) the modificationprofiles associated with the user. As an example, if the monitoringindicates that the second eye's gaze direction is directed at the firstposition, the first modification profile (e.g., its modificationparameters) may be used to modify the video stream data to generate themodified video stream data to be displayed to the user's first eye. Asanother example, if the monitoring indicates that the second eye's gazedirection is directed at the second position, the second modificationprofile (e.g., its modification parameters) may be used to modify thevideo stream data to generate the modified video stream data for theuser's first eye, and so on. In this way, for example, the foregoingaccounts for the typically dynamic nature of double vision (e.g., thedouble vision increases or decreases towards one or more gazes). Forexample, if the user has an issue in moving the user's right pupil awayfrom the user's nose (e.g., toward to edge of the user's face), then theuser's double vision may increase when the user is looking to the rightand may decrease when the user is looking to the left. As such, theuser's pupils, the user's line of sight, or other eye-relatedcharacteristics may be monitored to provide appropriate correction byapplying the appropriate modification profile specific to the user'sreal-time eye-related characteristics to the live video stream data.

In some embodiments, a vision test may be performed to assess binocularvision of a user. In some embodiments, a wearable device may be used toperform the binocular vision test. As an example, one or more stimulimay be presented on a user interface of each wearable device display foran eye of the user, where the number or type of stimuli presented on oneuser interface is different from the number or type of stimuli presentedon the other user interface (e.g., different number of stimuli on eachuser interface, at least one stimuli on one user interface having adifferent color or pattern than the stimuli in the other user interface,etc.). Alternatively, in some scenarios, the number or type of stimulipresented on both user interface is the same. Testing subsystem 122 maydetermine whether the user has double vision based on a user indicationof the number or types of stimuli that the user sees.

In one use case, with respect FIG. 25J, the binocular vision test mayinvolve a user wearing a wearable device having displays 2522 a and 2522b (or viewing such displays 2522 a and 2522 b via another device), whereeach display 2522 is configured to present one or more stimuli or otherprovide other presentations to a respective eye of the user. As anexample, stimuli 2524 a and 2524 b (e.g., green dots) may be presentedto one eye of the user on display 2522 a, and stimuli 2526 a, 2526 b,and 2526 c (e.g., red dots) may be presented to the other eye of theuser on display 2522 b. With respect to FIG. 25K, testing subsystem 122may determine that the user is seeing binocular single vision (and,thus, does not have double vision) based on a user indication that theuser sees 4 dots. Additionally, or alternatively, testing subsystem 122may determine or confirm that the user is seeing binocular single visionbased on a user indication that that the user is seeing one green dot(e.g., stimulus 2524 a), two red dots (e.g., stimuli 2526 a and 2526 c),and one mixed color dot (e.g., mixed stimulus 2528 from the combinationof stimuli 2524 b and 2526 b). On the other hand, with respect to FIG.25L, testing subsystem 122 may determine that the user has double vision(e.g., diplopia) based on a user indication that the user sees 5 dots.Additionally, or alternatively, testing subsystem 122 may determine orconfirm that the user has double vision based on a user indication thatthat the user is seeing two green dot (e.g., stimuli 2524 a and 2524 b)and three red dots (e.g., stimuli 2526 a, 2526 b, and 2526 c).

In some embodiments, testing subsystem 122 may monitor one or moreeye-related characteristics related to eyes of a user during visual testpresentation via two or more user interfaces (e.g., on two or moredisplays) and determine whether the user has double vision based on theeye-related characteristics occurring during the visual testpresentation in an autonomous manner. In some embodiments, testingsubsystem 122 may determine an extent of the user's double vision basedon such eye-related characteristics (e.g., by measuring the deviation ofone or more eyes as described herein) and generate one or moremodification profiles to correct for the double vision in an autonomousmanner. As an example, a wearable device may include a pupil and line ofsight tracker to detect the gaze direction of one or more eyes of theuser or other eye-related characteristics. Based on the gaze direction(or the other eye-related characteristics, testing subsystem 122 maydetermine the number of points on which the user fixated (e.g., by usingthe detected gaze directions to see whether the user fixated onpositions corresponding to the presented stimuli). In one use case, withrespect to FIG. 25J, if it is determined that the user fixated on fourpoints (e.g., points corresponding to stimuli 2524 a, 2526 a, 2526 c,and 2528 shown in FIG. 25K), testing subsystem 122 may determine thatthe user does not have double vision. If it is determined that the userfixated on five points (e.g., points corresponding to stimuli 2524 a,2524 b, 2526 a, 2526 b, and 2526 c shown in FIG. 25L), testing subsystem122 may determine that the user has double vision.

As a further example, in response to determining that the user hasfixated on a particular point (e.g., corresponding to the presentedstimuli or their respective display positions), testing subsystem 122may mitigate the impact of the corresponding stimuli and increase thecount of the number of stimuli that the user sees. As an example, thecorresponding stimuli may be removed from the visual test presentation(e.g., the corresponding stimuli will disappear and the remainingstimuli may continue to be presented) or modified to reduce its impact(e.g., by decreasing the brightness or other intensity level of thestimuli). As another example, the other stimuli may be modified toincrease its impact (e.g., by increasing the brightness or otherintensity level of the other stimuli), thereby reducing the relativeimpact of the corresponding stimuli. As such, the user's eyes willinstinctively move and fixate on one or more points corresponding to theremaining stimuli. With respect to FIG. 25K, for example, stimuli 2524 band 2526 b (represented by mixed stimuli 2528) will be removed when theuser's eyes fixate on the positions corresponding to stimuli 2524 b and2526 b. On the other hand, with respect to FIG. 25L (where the user hasdouble vision), stimuli 2524 b and 2526 b will be removed at twodifferent times because the user will not fixate on the same relativeposition when the user is looking at stimuli 2524 b or 2526 b. Testingsubsystem 122 may continue to remove stimuli and increase the count (ofthe number of stimuli that the user sees) in response to each of theuser's fixations on the corresponding points. When all of the stimulihave been removed or other threshold has been satisfied, testingsubsystem 122 may provide the number of stimuli that the user sees.

In some embodiments, based on eye-related characteristics occurringduring a visual test presentation, testing subsystem 122 may determinewhether the user has stereopsis or an extent of the user's stereopsis.As an example, testing subsystem 122 may cause one or more stimuli to bepresented at one or more positions on one or more user interfaces andperform such stereopsis determinations or other visual defectinformation based on the eye-related characteristics in an autonomousmanner. In one use case, with respect to FIG. 25M, the visual testpresentation may involve a user wearing a wearable device havingdisplays 2542 a and 2542 b (or viewing such displays 2542 a and 2542 bvia another device), where each display 2542 is configured to presentone or more stimuli or other provide other presentations to a respectiveeye of the user.

As shown in FIG. 25M, one or more icons 2544 or other stimuli may bepresented on each display 2542, where one or more pairs of the icons2544 are presented at corresponding positions on both displays 2542, andat least one pair of the icons 2544 is presented at slightly differentpositions on displays 2542 a and 2542 b. In particular, in FIG. 25M, thearrangement of the icons 2544 on both displays 2542 are the same, exceptthat the icon 2544 in the second row and third column on display 2542 bis shift slightly up and to the right (as shown by indicator 2546). To auser without binocular double vision and stereopsis, the slightdifference will cause the icon pair to appear as a three-dimensionalicon to the user, and all the other icons 2544 will appear astwo-dimensional icons to the user. As such, the user will instinctivelymove and fixate on the three-dimensional icon. Based on a determinationthat the individual has fixated on the three-dimensional icon (e.g.,within a predetermined threshold amount of time), testing subsystem 122may determine that the user does not have stereopsis. As an example,testing subsystem 122 may detect that the gaze direction of one or moreeyes of the user has changed upon the stimuli presentation and iscurrently directed toward the area at which the corresponding icons 2544are presented on their respective displays 2542.

If, however, the user has stereopsis, the slight difference may notcause the icon pair to appear as a three-dimensional icon to the user,and the user likely will not fixate on the corresponding area at whichthe icon pair are presented on their respective displays 2542. Based onthis lack of fixation (e.g., within the predetermined threshold amountof time), testing subsystem 122 may determine that the user hasstereopsis.

In a further use case, with respect to FIG. 25M, the amount of thedisparity between the two icons 2544 in the second row and third columnicon 2544 may be modified to determine an extent of the user'sstereopsis. As an example, the icon 2544 (in the area shown by indicator2546) may be initially shifted up or to the right such that thedisparity in the positions of the icon 2544 on display 2542 b and itscorresponding icon 2544 on display 2542 a is a minimal amount. If theuser does not fixate on the corresponding area at which the icon pairare presented, the icon 2544 on display 2542 b may be shifted up or tothe right again such that the disparity in the positions between the twoicons 2544 is slightly greater. The positional disparity increase may berepeated until the user fixates on the corresponding area or until apositional disparity threshold has been reached. Testing subsystem 122may use the positional disparity amount (or the number of times that theshifting operation is performed) to measure the extent of the user'sstereopsis.

In another use case, with respect to FIG. 25N, the stimuli presentationduring the visual test presentation may be provided in the form ofrandomly generated noise. In FIG. 25N, the stimuli presented on display2562 a and the stimuli presented on display 2562 b are the same, exceptthat the set of blocks (e.g., pixels) within the area shown by indicator2564 is shifted to the right by five units (e.g., pixels) in display2562 b (as compared to the same set of blocks in display 2562 a). Aswith the foregoing use case with respect to FIG. 25M, the slightdifference will cause the set of blocks to appear as a three-dimensionalobject (or otherwise be noticeable) to a user without binocular doublevision and stereopsis, resulting in the user quickly fixating on thethree-dimensional object. Based on a determination that the individualhas fixated on the three-dimensional object, testing subsystem 122 maydetermine that the user does not have stereopsis. If, however, the userhas stereopsis, the slight difference may not cause the set of blocks tobe noticeable to the user, and the user will not fixate on thecorresponding area at which the set of blocks are presented on theirrespective displays 2562. Based on this lack of fixation, testingsubsystem 122 may determine that the user has stereopsis.

In some embodiments, with respect to FIG. 1A, visioning subsystem 124may facilitate an increase in a field of view of a user via combinationof portions of multiple images of a scene (e.g., based on feedbackrelated to a set of stimuli displayed to the user). As an example, FIG.26 illustrates a representation of a normal binocular vision for asubject, where a monocular image from the left eye 2602 and from theright eye 2604 are combined into a single perceived image 2606 having amacular central area 2608 and a peripheral visual field area 2610surrounding the central area 2608. In some cases, however, a subject mayhave a tunnel vision condition, wherein the peripheral area 2610 is notvisible to the subject, as shown in the representation in FIG. 27. Asshown, for these cases, one or more objects do not appear within a fieldof view, resulting in a peripheral defect 2612 in the area 2610, whereobjects within the area 2610 are not seen by the subject. Thus, forexample, visioning subsystem 124 may combine portions of multiple imagesof a scene (e.g., common and divergent regions of such images) toincrease the field of view of the subject.

In some embodiments, visioning subsystem 124 may obtain a plurality ofimages of a scene (e.g., images obtained via one or more cameras atdifferent positions or orientations). Visioning subsystem 124 maydetermine a region common to the images, and, for each image of theimages, determine a region of the image divergent from a correspondingregion of at least another image of the images. In some embodiments,visioning subsystem 124 may generate or display an enhanced image to auser based on the common region and the divergent regions. As anexample, the common region and the divergent regions may be combined togenerate the enhanced image to include a representation of the commonregion and representations of the divergent regions. The common regionmay correspond to respective portions of the images that have the sameor similar characteristics as one another, and each divergent region maycorrespond to a portion of one of the images that is distinct from allthe other corresponding portions of the other images. In one scenario, adistinct portion of one image may include a part of the scene that isnot represented in the other images. In this way, for example, thecombination of the common region and the divergent region into anenhanced image increase the field of view otherwise provided by each ofthe images, and the enhanced image may be used to augment the user'svisual field. In one use case, the common region may be any portion ofat least one of the images of the left eye 2602 or the right eye 2604between any of two of the four vertical dotted lines indicated in FIG.27 for each such image. In another use case, with respect to FIG. 27,one of the divergent regions may be any portion of the image of the lefteye 2602 to the left of the left-most vertical dotted line for thatimage. Another one of the divergent regions may be any portion of theimage of the right eye 2604 to the right of the right-most verticaldotted line for that image.

In some embodiments, the common region is a region of at least one ofthe images that corresponds to a macular region of a visual field of aneye (or other central region of the visual field of the eye) or to aregion within the macular region. In some embodiments, each of thedivergent regions is a region of at least one of the images thatcorresponds to a peripheral region of a visual field of an eye or to aregion within the peripheral region. As an example, with respect to FIG.27, the common region may be (i) the portion of the image correspondingto the macular region of the left eye 2602 or (ii) the portion of theimage corresponding to the macular region of the right eye 2604 (e.g.,given that both such portions are common to both images). As anotherexample, the common region may be the respective portions of the imagescorresponding to a common region within the macular regions of the lefteye 2602 and right eye 2604. As a further example, based on the commonregion and the divergent regions, the image 2606 is generated to havethe macular central area 2608 and the peripheral visual field area 2610surrounding the central area 2608.

In some embodiments, visioning subsystem 124 may determine a regioncommon to a plurality of images of a scene (e.g., captured via awearable device of the user), and, for each image of the images,determine a region of the image divergent from a corresponding region ofat least another image of the images. Visioning subsystem 124 mayperform shifting of each image of the images and generate, subsequent tothe performance of the shifting, an enhanced image based on the commonregion and the divergent regions. In some embodiments, the shifting ofeach of the images may be performed such that (i) a size of the commonregion is modified (e.g., increased or decreased) or (ii) a size of atleast one of the divergent regions is modified (e.g., increased ordecreased). In one scenario, the size of the common region may beincreased as result of the shifting. In another scenario, the size of atleast one of the divergent regions is decreased as a result of theshifting.

As an example, the defect in FIG. 27 may be corrected using a shiftingimage correction technique. In one use case, with respect to FIG. 28,each of two visual field cameras (e.g., of a wearable device) maycapture a monocular image 2802 and 2804, respectively (e.g., where eachmonocular image is different as it's capturing the visual scene from aslightly different (offset) position). The two captured monocular images2802, 2804 are then shifted toward each other in the vision correctionframework resulting in images 2802′ and 2804′. As shown in FIG. 28, therespective areas (e.g., a common region) of the two images 2802 and 2804between the left-most vertical dotted line and the right-most verticaldotted line for each image 2802 and 2804 (is larger than the respectiveareas (e.g., a common region) between the two images 2802′ and 2804′between the left-most vertical dotted line and the right-most verticaldotted line for each image 2802′ and 2804′. As such, the common regionis decreased in size subsequent the shifting. On the other hand, thedivergent regions have increased in size subsequent the shifting (e.g.,the area left of the left-most vertical dotted line for image 2802 vs.the area left of the left-most vertical dotted line for image 2802′, andthe area right of the right-most vertical dotted line for image 2804 vs.the area right of the right-most vertical dotted line for image 2804′).

As a further example, these two shift images are then combined togenerate a binocular image 2806 that captures the full periphery of thevisual scene. For spectacles device having monitor displays, eachdisplay may display the corrected binocular image 2806 to the subject.In some use cases, for example, this shifting transformation can be usedto increase the field of view of a subject by 5%, 10%, 15%, 20%, ormore, without producing double vision effects for the subject.

In some embodiments, visioning subsystem 124 may determine a regioncommon to a plurality of images of a scene (e.g., captured via awearable device of the user), and, for each image of the images,determine a region of the image divergent from a corresponding region ofat least another image of the images. Visioning subsystem 124 mayperform resizing of one or more regions of the images and generate,subsequent to the performance of the resizing, an enhanced image basedon the common region and the divergent regions. In some embodiments,visioning subsystem 124 may perform resizing of one or more regions ofthe images such that an extent of any resizing of the common region isdifferent than an extent of any resizing of at least one of thedivergent regions. In some embodiments, the resizing may be performedsuch that a percentage change in size of the common region representedin a first region of the enhanced image is greater than or less than apercentage change in size of at least one of the divergent regionsrepresented in a second region of the enhanced image. As an example, thepercentage change in size of at least one of the divergent regions maybe zero, and the percentage change in size of the common region may begreater than zero. As another example, the percentage change in size ofat least one of the divergent regions may be greater than zero, and thepercentage change in size of the common region may be zero.

In one scenario, with respect to FIG. 29, captured monocular images 2902and 2904 are resized only in peripheral areas, while keeping the macularcentral area (central 20 degrees) unchanged, resulting in correctedimages 2902′, 2904′. Such resizing transformation will preserve thevisual acuity in the center while expanding the visual field. As shownin FIG. 29, a combined binocular image 2906 captures the objects in theperiphery that were missed before, and at the same time, keeps thedetails of the central macular area. The peripheral objects are clearlynoticed by the subject even after resizing them, as the peripheralvision is not as sensitive as the central one. In some use cases, forexample, shrinking of up to 20% of the image size can be performedwithout producing double vision effects for the subject. In variousembodiments, resizing of a peripheral region may be performedadditionally or alternatively to resizing of a central area. Forexample, peripheral regions may be resized to the sizes of theperipheral regions while retaining the size of the macular central area(e.g., for glaucoma patients). In another scenario, for patients withmacular degeneration, the peripheral vision may be left intact (e.g.,with no resizing), and the central area may be resized to reduce thesize of the central area. The enhanced image (e.g., the binocular image)may then be generated to include the resized central area.

In some embodiments, visioning subsystem 124 may determine a regioncommon to a plurality of images of a scene (e.g., captured via awearable device of the user), and, for each image of the images,determine a region of the image divergent from a corresponding region ofat least another image of the images. Visioning subsystem 124 mayperform a fisheye transformation, a conformal mapping transformation, orother transformation on the common region and generate, subsequent tothe performance of the transformation, an enhanced image based on thecommon region and the divergent regions. In some embodiments, visioningsubsystem 124 may perform the fisheye transformation, the conformalmapping transformation, or other transformation on a region of theenhanced image (that includes the common region).

As an example, the fisheye transformation may be performed on a regionto modify a radical component of the images in accordance with:

r _(new) =r+αr ³, where α is a constant.

As another example, the conformal mapping transformation may beperformed on a region to modify a radial component of the images inaccordance with:

r _(new) =r ^(β), where β is a constant power of the radial componentand β>1

In some embodiments, visioning subsystem 124 may modify at least one ofa plurality of images of a scene by moving one or more objects in theimage (e.g., prior to generating an enhanced image based on common anddivergent regions of the images). As an example, with respect to FIG.30, for patients with far peripheral defect in one eye, a missing object3002 in a visual field 3004 of the defective eye can be transferreddigitally to a mid-peripheral field region 3006 of the visual field3004, while other visual field 3008, that of the healthy eye, wouldotherwise cover this area, meaning that the combined binocular image3010 displays the missing object 3002 within an intact visual field. Thesubject may notice visual confusion in the area, but the subject canadapt to isolate information in this area of the visual field accordingto a moving object or the changing environment.

In some embodiments, visioning subsystem 124 may determine one or moredefective visual field portions of a visual field of a user (e.g., inaccordance with one or more techniques described herein). In someembodiments, visioning subsystem 124 may determine a region common to aplurality of images of a scene (e.g., captured via a wearable device ofthe user), and, for each image of the images, determine a region of theimage divergent from a corresponding region of at least another image ofthe images. Visioning subsystem may generate an enhanced image based onthe common and divergent regions of the images such that at least one ofthe common or divergent regions in the enhanced image do not overlapwith one or more of the defective visual field portions.

In some embodiments, visioning subsystem 124 may detect an object in adefective visual field portion of a visual field of a user and cause analert to be displayed. As an example, after correcting for defectivevisual field portion of a visual field of a user (e.g., via one or moretechniques described herein), visioning subsystem 124 may monitor theremaining regions that were not corrected to detect one or more objects(e.g., safety hazards or other objects) and generate alerts (e.g.,visual or audible alerts) indicating the objects, locations of theobjects, the size of the objects, or other information related to theobjects. In one use case, for a patient with irregular or multi-regiondefective visual field, the produced modification profile might stillnot be optimal in fitting the acquired field of view into the intactregions of the patient's visual field. Therefore, to maximize thepatient's safety while moving, automatic video tracking algorithms maybe implemented to detect objects that are in one of the detective visualfield portions. Such objects may include moving objects (e.g., movingcar) or other objects in the defective visual field portions of thepatient's visual field.

In some embodiments, visioning subsystem 124 may generate a predictionindicating that an object will come in physical contact with a user andcause an alert to be displayed based on the physical contact prediction(e.g., an alert related to the object is displayed on a wearable deviceof the user). In some embodiments, visioning subsystem 124 may detect anobject (e.g., in or predicted to be in a defective visual field portionof a visual field of a user) and cause the alert to be displayed basedon (i) the object being in or predicted to be in the defective visualfield portion, (ii) the physical contact prediction, or (iii) otherinformation. In some embodiments, visioning subsystem 124 may determinewhether the object is outside (or not sufficiently in) any image portionof an enhanced image (displayed to the user) that corresponds to atleast one visual field portions satisfying one or more vision criteria.In one use case, no alert may be displayed (or a lesser-priority alertmay be displayed) when the object is determined to be within (orsufficiently in) an image portion of the enhanced image that correspondsto the user's intact visual field portion (e.g., even if the object ispredicted to come in physical contact with the user). On the other hand,if the object in the defective visual field portion is predicted to comein physical contact with the user, and it is determined that the objectis outside (or not sufficiently in) the user's intact visual fieldportion, an alert may be displayed on the user's wearable device. Inthis way, for example, the user can rely on the user's own intact visualfield to avoid incoming objects within the user's intact visual field,thereby mitigating the risk of dependence on the wearable device (e.g.,through habit forming) for avoidance of such incoming objects. It shouldbe noted, however, that, in other use cases, an alert related to theobject may be displayed based on the physical contact predictionregardless of whether the object is within the user's intact visualfield.

As an example, with respect to FIG. 10, for the uncompensated blindfield 1006, at blocks 1012 and 1014, pupil tracking or other visiontracking (e.g., using inward directed image sensors) video tracking of amoving object in the visual field (e.g., through outward directed imagesensors such as external cameras) may be used to detect safety hazardsin regions of blind spots or that are moving into the regions of blindspots. In one use case, visioning subsystem 124 may compare the positionof the safety hazard to a mapped visual field with defects (e.g., asmeasured in a testing mode) to detect when the safety hazard is inregions of blind spots or when the safety hazard is moving into suchregions.

As another example, after correcting for defective visual field portionof a visual field of a user (e.g., via one or more techniques describedherein), visioning subsystem 124 may monitor the remaining regions thatwere not corrected to detect any safety hazard (e.g., in real-time)approaching the user from such regions. If such detected safety hazardsare predicted to come in physical contact with the user or come within athreshold distance of the user (e.g., one feet, two feet, or otherthreshold distance) (as opposed to passing by the user by at least thethreshold distance of the user), visioning subsystem 124 may generate analert related to the detected safety hazard (e.g., a visual alertdisplayed on a region seeable by the user, an audible alert, etc.).

In one use case, video signals (e.g., a live video stream) acquired fromone or more cameras of a wearable device of a user will be preprocessedand filtered to remove residual noise effects. In some cases, the searchregion may be limited to the blind spots of the user or other defectivevisual field portions (e.g., that fail to satisfy one or more visioncriteria). The limiting of the search region, for example, may reducethe amount of computational resources required to detect objects in thesearch region or generate related alerts or increase the speed of suchdetection or alert generation.

In some cases, two successive frames from a live video stream may besubtracted from one another to detect motion of one or more objects. Asan example, occurrence of motion may be stored on a first delta frame(e.g., delta frame 1), and the first delta frame may be used to enablevisualization of the moving objects and cancelling the stationarybackground. Another two successive frames from the live video stream maybe subtracted from one another to produce a second delta frame (e.g.,delta frame 2). The second delta frame may also be used to enablevisualization of the moving objects and cancelling the stationarybackground. In further cases, comparison between the first and seconddelta frames may be performed. If a moving object is increasing in sizeas detected by subtracting the first delta frame and the second deltaframe from one another, then the object may be determined to be gettingcloser. If the increase in size exceeds a predetermined threshold size,then the alert will be issued to the user (e.g., a visual alertdisplayed on a region seeable by the user, an audible alert, etc.).

In some embodiments, configuration subsystem 112 may store predictionmodels, modification profiles, visual defect information (e.g.,indicating detected visual defects of a user), feedback information(e.g., feedback related to stimuli displayed to users or otherfeedback), or other information at one or more remote databases (e.g.,in the cloud). In some embodiments, the feedback information, the visualdefect information, the modification profiles, or other informationassociated with multiple users (e.g., two or more users, ten or moreusers, a hundred or more users, a thousand or more users, a million ormore users, or other number of users) may be used to train one or moreprediction models. In one use case, where a prediction model beingtrained is a neural network or other machine learning model, modelmanager subsystem 114 may provide as input to the machine learning model(i) stimuli information (e.g., indicating a set of stimuli and theirassociated characteristics, such as intensity levels, locations at whicha stimuli is to be displayed, etc.) and (ii) feedback information (e.g.,indicating feedback related to the set of stimuli) to cause the machinelearning model to predict visual defect information, modificationprofiles, or other outputs. Model manager subsystem 114 may providereference information (e.g., visual defect information or modificationprofiles determined to be accurate with respect to the provided stimuliand feedback information) to the machine learning model. The machinelearning model may assess its predicted outputs (e.g., predicted visualdefect information, predicted modification profiles, etc.) against thereference information and update its configurations (e.g., weights,biases, or other parameters) based on its assessment of its predictedoutputs. The foregoing operations may be performed with additionalstimuli information (e.g., displayed to other users), additionalfeedback information (e.g., the other users' feedback related to thestimuli displayed to them), and additional reference information tofurther train the machine learning model (e.g., by providing suchinformation as input and reference feedback to train the machinelearning model, thereby enabling the machine learning model to furtherupdate its configurations).

In another use case, where the machine learning model is a neuralnetwork, connection weights may be adjusted to reconcile differencesbetween the neural network's prediction and the reference information.In a further use case, one or more neurons (or nodes) of the neuralnetwork may require that their respective errors be sent backwardthrough the neural network to them to facilitate the update process(e.g., backpropagation of error). Updates to the connection weights may,for example, be reflective of the magnitude of error propagated backwardafter a forward pass has been completed.

In some embodiments, one or more prediction models may be trained orconfigured for a user or a type of device (e.g., a device of aparticular brand, a device of a particular brand and model, a devicehaving a certain set of features, etc.) and may be stored in associationwith the user or the device type. As an example, instances of aprediction model associated with the user or the device type may bestored locally (e.g., at a wearable device of the user or other userdevice) and remotely (e.g., in the cloud), and such instances of theprediction model may be automatically or manually synced across one ormore user devices and the cloud such that the user has access to thelatest configuration of the prediction model across any of the userdevices or the cloud. In one use case, upon detecting that a first useris using a wearable device (e.g., when the first user logs into theuser's account or is identified via one or more other techniques),configuration subsystem 112 may communicate with the wearable device totransmit the latest instance of a prediction model associated with thefirst user to the wearable device such that the wearable device hasaccess to a local copy of the prediction model associated with the firstuser. In another use case, if a second user is later detected to beusing the same wearable device, configuration subsystem 112 maycommunicate with the wearable device to transmit the latest instance ofa prediction model associated with the second user to the wearabledevice such that the wearable device has access to a local copy of theprediction model associated with the second user.

In some embodiments, multiple modification profiles may be associatedwith the user or the device type. In some embodiments, each of themodification profiles may include a set of modification parameters orfunctions to be applied to live image data for a given context togenerate an enhanced presentation of the live image data. As an example,the user may have a modification profile for each set of eyecharacteristics (e.g., a range of gaze directions, pupil sizes, limbuspositions, or other characteristics). As further example, the user mayadditionally or alternatively have a modification profile for each setof environmental characteristics (e.g., a range of brightness levels ofthe environment, temperatures of the environment, or othercharacteristics). Based on the eye characteristics or environmentalcharacteristics currently detected, the corresponding set ofmodification parameters or functions may be obtained and used togenerate the enhanced presentation of the live image data. In one usecase, upon detecting that a first user is using a wearable device (e.g.,when the first user logs into the user's account or is identified viaone or more other techniques), configuration subsystem 112 maycommunicate with the wearable device to transmit the modificationprofiles associated with the first user to the wearable device such thatthe wearable device has access to a local copy of the modificationprofiles associated with the first user. In another use case, if asecond user is later detected to be using the same wearable device,configuration subsystem 112 may communicate with the wearable device totransmit the modification profiles associated with the second user tothe wearable device such that the wearable device has access to a localcopy the modification profiles associated with the second user.

In some embodiments, visual field locations of a user's field of viewmay be tested for visual defects. The visual field locations may be, forexample, locations in a grid that covers the user's field of view or aportion of the user's field of view. For example, certain vision testsmay test a certain number of visual field locations (e.g., 76 or othernumber of locations) of the user's field of view. The number of visualfield locations may depend on a field of view of the user, a visualdefect for which the user is being tested, the particular type of visualtest being run, or any other factors. In some embodiments, stimuli maybe presented at user interface locations of a wearable device (e.g.,spectacles device 170, as shown in FIGS. 1C and 1D) that correspond tothe visual field locations of the user's field of view. A stimulus ateach visual field location may be tested over a series ofcharacteristics (e.g., 50 different contrast levels) in order todetermine a threshold characteristic under which the user can see thestimulus. In some embodiments, a threshold characteristic may be acharacteristic above which a user can see the stimulus and below whichthe user cannot see the stimulus (or vice versa). For example, testingsubsystem 122 may perform a test for each visual field location in auser's field of vision. In some embodiments, testing subsystem 122 maydisplay each stimulus to the user in a descending or ascending order ofcharacteristics (e.g., brightness levels, contrast levels, saturationlevels, sharpness levels, or another range of characteristics). Forexample, testing subsystem 122 may present each stimulus at a range ofcontrast levels.

Testing every visual field location under each characteristic requirestime and resources, which may be reduced via one or more techniquesdescribed herein. In some embodiments, a vision test (e.g., configuredto test a plurality of visual field locations of a user) may befacilitated by obtaining a predicted characteristic with respect to eachuser interface location of a plurality of user interface locations(corresponding to the visual field locations) and setting an initialstimuli characteristic for each such user interface location based onthe predicted characteristic for the user interface location. As anexample, the predicted characteristic for a given user interfacelocation may be a characteristic under which the user is predicted tosee a stimulus at the given user interface location. The initialstimulus characteristic for the given user interface location may be (i)the predicted characteristic (e.g., a threshold characteristic underwhich a user is predicted to see the stimulus at the given userinterface location), (ii) a characteristic adjacent or proximate thepredicted characteristic, or (iii) other characteristic. In this way,for example, such prediction-based setting of initial stimulicharacteristics reduces the number of times a stimulus is required to bepresented at each user interface location, thereby decreasing the amountof time of vision testing and the amount of resources related to suchvision testing (e.g., computational or other resources).

In some embodiments, one or more prediction models may be used todetermine a characteristic under which to initially present a stimulusat a given location. For example, a prediction model (e.g., neuralnetwork) may be trained on training sets and updated using feedback fromusers. Prediction models may increase the efficiency of vision testingby reducing the number of characteristics under which each stimulus mustbe tested. For example, predicted characteristics may allow eachlocation to be tested with a minimal amount of testing time bypresenting a stimulus at that location only twice under two differentcharacteristics (e.g., at a contrast level below or at a thresholdlevel, at a contrast level below and above the threshold level, etc.).As such, time and resources related to vision testing may be saved(e.g., computational resources or other resources).

In some embodiments, a prediction model (e.g., one or more instances ofthe prediction model) may be configured based on visual field datasetsthat incorporate different types of peripheral defects. For example, atraining dataset may include visual field defect patterns so that theprediction model is trained to identify such visual field defectpatterns during the testing process. In some embodiments, during auser's visual test, such a pre-configured prediction model (e.g., aninstance of such prediction model) may additionally be trained usinginitial visual test results from the user to personalize the predictionmodel for the user. For example, several visual field locations andthreshold characteristics under which the user saw stimuli at thosevisual field locations may be input into the training model. In thisexample, a sample of initial results from the vision test (e.g., for oneor more stimuli) may improve the configuration of the prediction modelto generate improved predictions. In this way, for example, even wherethe training dataset used to produce the initial pre-configuredprediction model is relatively small, the use of the user's own initialvisual test results to further train (and personalize) the predictionmodel will minimize the accuracy gap between the prediction model'spredicted characteristics and those of a model created using asubstantially larger training dataset.

Once the prediction model is configured (e.g., initially configured orfurther updated), the prediction model may generate one or morepredictions for one or more visual field locations. In some embodiments,a prediction may be a first predicted characteristic (e.g., 43%contrast) under which the user is predicted to see a first stimulus at afirst user interface location (that corresponds to a first visual fieldlocation). The vision test may then include presenting the firststimulus with the first predicted characteristic (e.g., 43% contrast)under which the user is predicted to see the first stimulus. In someembodiments, prior to presenting using the first predictedcharacteristic for the first stimulus, the vision test may includepresenting the first stimulus with an adjacent characteristic (e.g., 42%contrast) under which the user is predicted to not see the firststimulus. In some embodiments, the generated prediction may be acharacteristic which should be tested first during the vision test. Forexample, the generated prediction may be a characteristic adjacent to acharacteristic (e.g., in the range of characteristics) under which theuser is predicted to see the first stimulus at the first user interfacelocation. The vision test may include presenting the first stimulus withthe characteristic output by the prediction model (e.g., 42% contrast)under which the user is predicted to not see the first stimulus,followed by presenting the first stimulus with a characteristic adjacentto the characteristic output by the prediction model (e.g., 43%contrast) under which the user is predicted to see the stimulus.

In some embodiments, testing subsystem 122 may cause one or more stimulito be presented to the user. As an example, the stimuli may be presentedon a display (e.g., of a wearable device) or projected onto a retina ora cornea of the user to determine defects affecting the retina or thecornea. In some embodiments, each stimulus may be presented (e.g., viatesting subsystem 122) under a range of characteristics relating toretinal sensitivity. For example, the range of characteristics mayinclude brightness levels, contrast levels, saturation levels, sharpnesslevels, or another range of characteristics. For example, with respectto contrast levels, the stimuli may differ in contrast levels withrespect to each other and with respect to a baseline contrast level by acertain amount (e.g., at least 20 dB). In some cases, the stimuli maydiffer in contrast levels with respect to each other and with respect toa baseline contrast level by a different amount (e.g., at least 30 dB).In some cases, testing subsystem 122 may, in the testing mode, instructa wearable spectacles device to display the set of testing stimuli tothe user in a descending or ascending contrast.

As an example, with respect to FIG. 44A, a stimulus 4402 (e.g., thefirst stimulus) is presented at a first user interface location under afirst characteristic (e.g., 42% contrast) during a visual testpresentation 4400. As shown in FIG. 44B, stimulus 4402 (e.g., the firststimulus) is presented at the first user interface location under asecond characteristic (e.g., 43% contrast). As described above, FIG. 44Bmay show the stimulus presented under the first predictedcharacteristic, as output by the prediction model (e.g., acharacteristic under which the user is predicted to see stimulus 4402).In this example, testing subsystem 122 may subsequently present stimulus4402 as shown in FIG. 44A (e.g., which the user is predicted to not see)in order to confirm that the first predicted characteristic is thethreshold characteristic for the first user interface location. In someembodiments, the stimulus may be presented under additionalcharacteristics (e.g., 41% contrast, 40% contrast, etc.) until athreshold characteristic is reached.

In some embodiments, testing subsystem 122 may obtain feedback from auser in response to presentation of one or more stimuli to the user(e.g., the sets of stimuli 4402-4412, as shown in FIGS. 44A and 44B) anduse such feedback to update the prediction model. As an example, thefeedback may include an indication of a response of the user to one ormore stimuli or an indication of a lack of response of the user to suchstimuli. The response (or lack thereof) may relate to an eye movement, agaze direction, a pupil size change, or a user modification of one ormore stimuli, or other user input (e.g., the user's reaction or otherresponse to the stimuli). As another example, the feedback may includean eye image captured during the visual test presentation. The eye imagemay be an image of a retina of the eye (e.g., the overall retina or aportion thereof), an image of a cornea of the eye (e.g., the overallcornea or a portion thereof), or another eye image. In some embodiments,the feedback may indicate a threshold characteristic of the range ofcharacteristics under which the user sees each stimulus. For example,the threshold characteristic under which the user is able to see astimulus at a given user interface location may be different from apredicted characteristic for the user interface location. The predictionmodel may thus update its configurations based on its assessment of itsprediction (e.g., the predicted characteristic) and feedback information(e.g., threshold characteristic under which the user sees the stimulusat the user interface location, as indicated by the user feedback).

In some embodiments, testing subsystem 122 may use the updatedprediction model to generate a second predicted characteristic underwhich the user is able to see a second stimulus at a second userinterface location. In some embodiments, the second stimulus may bepresented under the second predicted characteristic (e.g., secondpredicted contrast level), as output by the updated prediction model.Based on feedback from the user, testing subsystem 122 may present thesecond stimulus with a second characteristic (e.g., an adjacent contrastlevel, a higher contrast level, a lower contrast level, etc.) untilfeedback from the user indicates that a threshold characteristic isreached. In some embodiments, the prediction model may be continuouslyupdated based on each predicted characteristic and reference feedback(e.g., feedback from the user).

In some embodiments, testing subsystem 122 may use the updatedprediction model to obtain a pattern for a second set of locations ofthe user interface. For example, testing subsystem 122 may identify avisual field defect pattern (e.g., as described above) based on thefeedback from the user in response to the first set of stimuli 4402, asshown in FIGS. 44A and 44B. The pattern for the second set of locationsof the user interface may therefore be based on the identified visualfield defect pattern. For example, the pattern may indicate predictedcharacteristics for user interface locations of the second set oflocations. In some embodiments, the pattern for the second set oflocations may be based on other aspects of the feedback from the user inresponse to the first set of stimuli 4402. In some embodiments, thesecond set of locations may include a different number of user interfacelocations than the first set of locations (e.g., a greater number ofuser interface locations).

FIG. 44C shows a second set of stimuli presented at a second set oflocations of the user interface 4400, in accordance with one or moreembodiments. For example, the second set of stimuli may include stimulus4406, stimulus 4408, stimulus 4410, and stimulus 4412. In someembodiments, each stimulus of the second set of stimuli 4402 may bepresented under at least one characteristic of the range ofcharacteristics. For example, the second set of stimuli 4402 may all bepresented under the same characteristic or under differentcharacteristics. In some embodiments, the pattern obtained by testingsubsystem 122 may indicate a predicted characteristic for each stimulusof the second set of stimuli. In this example, testing subsystem 122 maypresent each stimulus of the second set of stimuli based on the pattern.For example, testing subsystem 122 may present stimulus 4406 at a firstpredicted characteristic (e.g., 79% contrast), as indicated by thepattern. Testing subsystem 122 may then present stimulus 4406 under asecond characteristic (e.g., 78% contrast or 80% contrast, depending onwhether the user was able to see stimulus 4406 under the firstcharacteristic), and so on. Testing subsystem 122 may continue to teststimulus 4406 under various characteristics until a thresholdcharacteristic is reached. For example, testing subsystem 122 maydetermine that the threshold characteristic for stimulus 4406 is 76%contrast.

In some embodiments, testing subsystem 122 may proceed with testing thenext stimulus of the second set of stimuli (e.g., stimulus 4408) basedon a predicted characteristic under which the user is predicted to seestimulus 4408 (e.g., based on the pattern). Testing subsystem 122 maysubsequently present stimulus 4408 under a first characteristic, whichmay be the same as or different than the first characteristic used topresent stimulus 4406. Testing subsystem 122 may continue testingstimulus 4408 under various characteristics until a thresholdcharacteristic is reached for stimulus 4408. Testing subsystem 122 mayrepeat this process with the remaining stimuli of the second set ofstimuli (e.g., stimulus 4410 and stimulus 4412).

In some embodiments, each stimulus of the second set of stimuli 4402 maybe presented based on the pattern that was previously obtained. Forexample, the pattern may indicate threshold characteristics under whichthe user is predicted to see second set of stimuli presented at thesecond set of locations. In some embodiments, the prediction model maybe continuously updated based on each predicted characteristic andreference feedback (e.g., feedback from the user) for each stimulus. Forexample, testing subsystem 122 may use the feedback received in responseto the testing of each stimulus of the second set of stimuli in order toupdate the prediction model prior to testing subsequent stimuli of thesecond set of stimuli. In some embodiments, testing subsystem 122 mayupdate the prediction model after feedback is received for eachstimulus, after feedback is received for several stimuli, after feedbackis received for each set of stimuli, or at another time.

In some embodiments, based on feedback related to the stimuli (displayedto a user during the visual test) or other feedback, testing subsystem122 may determine light sensitivity, distortions, or other aberrationsrelated to one or more eyes of the user. In some embodiments, visioningsubsystem 124 may generate visual defect information for the user basedon feedback received from the user with respect to each stimuluspresented by testing subsystem 122. Testing subsystem 122 may determineone or more defective visual field portions of a visual field of a user(e.g., an automatic determination based on feedback related to thestimuli displayed to the user or other feedback). As an example, adefective visual field portion may be one of the visual field portionsof the user's visual field that fails to satisfy one or more visioncriteria (e.g., whether or an extent to which the user senses one ormore stimuli, an extent of light sensitivity, distortion, or otheraberration, or other criteria). In some cases, the stimuli may include agrid of stimuli corresponding to visual field locations within theuser's field of view. In some embodiments, a defective visual fieldportion may be a portion of the user's visual field which matches avisual field defect pattern. Defective visual field portions may includeregions of reduced vision sensitivity, regions of higher or loweroptical aberrations, regions of reduced brightness, or other defectivevisual field portions.

In some embodiments, a neural network may be used as at least part of aprediction model for facilitating vision testing. In some embodiments,visioning subsystem 124 may train a neural network to improve efficiencyand speed of vision testing. For example, visioning subsystem 124 mayprovide a first user interface location as input to a neural network(e.g., machine learning model 162, as shown in FIG. 1B). In someembodiments, the neural network may generate a first predictedcharacteristic under which the user is predicted to see a first stimulusat the first user interface location. In some embodiments, testingsubsystem 122 may present the first stimulus to the user under a rangeof characteristics. For example, testing subsystem 122 may present thefirst stimulus (e.g., corresponding to the first visual field location)with the first predicted characteristic (e.g., first predicted contrastlevel). Based on feedback from the user, testing subsystem 122 maypresent the first stimulus with a second characteristic (e.g., a higheror lower contrast level) until feedback from the user indicates that athreshold characteristic is reached. In some embodiments, visioningsubsystem 124 may provide the user feedback indicating that thethreshold characteristic has been reached as reference feedback to theneural network discussed above.

In some embodiments, the neural network may update its configurations(e.g., weights, biases, or other parameters) based on its assessment ofits prediction (e.g., first predicted threshold characteristic) andreference feedback information (e.g., threshold characteristic underwhich the user sees the first stimulus, as indicated by the userfeedback). As discussed herein, in some embodiments, connection weightsmay be adjusted to reconcile differences between the neural network'sprediction and the reference feedback. In some embodiments, the neuralnetwork may additionally or alternatively be trained on a visual fielddataset that incorporates different types of peripheral defects. Forexample, training datasets may include visual field defect patterns sothat the neural network is trained to identify such visual field defectpatterns during the testing process. In this way, for example, theneural network may be trained to generate better predictions.

In some embodiments, predictions of the neural network may be based uponrecognized visual field defect patterns. For example, the neural networkmay match initial results of a vision test to a pattern present incertain visual field defects. This may allow the neural network topredict characteristics under which the user will see stimuli to betested. In some embodiments, the updated neural network may generatepredicted characteristics that are similar to characteristics underwhich the user sees adjacent stimuli.

In some embodiments, multiple sets of predicted characteristics (e.g.,each set corresponding to a set of locations of the user interface) maybe obtained via a prediction model, and at least one set of the multiplesets may be selected to perform a vision test based on the predictedcharacteristics of the selected set(s) (e.g., via one or more techniquesdescribed herein).

In some embodiments, during one or more portions of a visual test, oneor more locations may be tested with one or more default startingcharacteristics (e.g., default starting contrast levels, brightnesslevels, contrast levels, sharpness levels, saturation levels, etc.)under which one or more stimuli may initially be presented to a user(e.g., after which the stimuli characteristic for a location may bedynamically adjusted if it is detected that the user is unable to see astimulus presented at the location). In this way, for example, initialvisual defect information or additional visual defect information may bedetermined and provided to a prediction model to obtain one or morepredicted characteristics for one or more additional locations to betested. As an example, when it is known that additional visual defectinformation is needed to increase accuracy of predictions for a visualfield region to a sufficient accuracy level, one or more defaultstarting characteristics may be used to initially present stimuli at oneor more locations (e.g., corresponding to locations in the visual fieldregion) to obtain and provide the visual defect information for thoselocations to the prediction model to increase the prediction model'saccuracy when predicting characteristics for one or more otherlocations.

In some embodiments, a default starting characteristic may be a startingcharacteristic configured to be used for every user in one or moreportions of the visual test. In some embodiments, a default startingcharacteristic may include (i) a starting characteristic specific to aleft eye or to a right eye, a starting characteristic specific to avirtual field region (e.g., central, paracentral, near-peripheral,mid-peripheral, far-peripheral, or other region), (ii) a startingcharacteristic specific to a demographic characteristic (e.g., age, sex,ethnicity, or other demographic), (iii) a starting characteristicspecific to a disease or other health condition (e.g., a particularheart disease, diabetes, or other health condition that the user isknown to have or known to be predisposed to have), or (iv) otherstarting characteristic. In one use case, based on one or more knowncharacteristics of the user (e.g., the particular eye to be tested, thevisual field regions to be tested, the demographic characteristics ofthe user, the health conditions of the user, etc.), the startingcharacteristics may be predetermined for a vision test prior toconducting the visual testing presentation of one or more stimuli forderiving (i) the threshold characteristics under which the user is ableto see such stimuli at the respective locations of the user interface(e.g., corresponding to locations of the user's visual field) or (ii)other visual defect information. In this way, for example, by beginningtesting of locations with such user-specific default startingcharacteristics, the number of times a stimulus is required to bepresented at each of the locations will be reduced overall, therebydecreasing the amount of time of vision testing and the amount ofresources related to such vision testing (e.g., computational or otherresources).

As an example, during a first portion of a visual test presentation, aninitial set of locations of a user interface may be tested using one ormore default starting characteristics (e.g., one or more defaultcontrast levels) to determine threshold characteristics (e.g., minimumcontrast levels) under which the user is able to see stimuli presentedat respective locations of the initial set. Feedback (e.g., indicatingsuch threshold characteristics) from the first portion of the visualtest presentation may be provided to a neural network, and the neuralnetwork may output one or more predicted characteristics for one or moreadditional locations to be tested. As a further example, during a secondportion of the visual test presentation, the predicted characteristicsfor the additional locations may then be used to test the user (e.g.,via one or more techniques described herein). The feedback (e.g.,indicating the respective threshold characteristics) from the secondportion of the visual test presentation may be provided a neural network(e.g., the same or different instance of the neural network from thefirst portion of the visual test presentation, a different neuralnetwork, etc.), and the neural network may output one or more predictedcharacteristics for one or more further locations to be tested. Thepredicted characteristics for the further locations may then be used totest the user during a third portion of the visual test presentation.The foregoing operations may be repeated for subsequent portions of thevision test until a threshold number of locations have been tested orall locations of a test set have been tested.

In one use case, with respect to FIGS. 46A-46B, during a first portionof a vision test, four locations 4602 on user interface 4600 (e.g.,corresponding to a location in each central quadrant of a visual fieldof a user) may be initially tested (e.g., without prior knowledge) withone or more default starting characteristics (e.g., a default startingcontrast level or other default starting characteristics), and feedbackfor the locations 4602 (e.g., indicating threshold characteristics underwhich the user is able to see stimuli at the respective locations 4602)may be obtained during the first portion of the vision test. Suchfeedback may be provided as a ground truth input to a neural network,and the neural network may output predicted characteristics for twelvelocations 4604 on user interface 4600. During a second portion of thevision test, the predicted characteristics for the locations 4604 may beused to test the user. The feedback for the locations 4604 (e.g.,indicating the respective threshold characteristics from the secondportion of the vision test) may be obtained during the second portion ofthe vision test. Such feedback may be provided as a ground truth inputto a neural network, and the neural network may output one or morepredicted characteristics for one or more further locations to be testedduring a subsequent portion of the vision test. The neural network usedfor the second portion of the vision test may, for example, be the sameinstance of the neural network from the first portion of the vision testor a different instance of the same neural network (or otherwise adifferent neural network) to accommodate or optimize for differentinput/output configurations.

In another use case, with respect to FIGS. 46C-46D, during a thirdportion of the vision test, four locations 4606 on user interface 4600(e.g., corresponding to a location in each paracentral quadrant of avisual field of a user) may be tested with one or more default startingcharacteristics, and feedback for the locations 4606 may be obtained.Such feedback for the locations 4606 along with the prior feedback(e.g., for one or more of the locations 4602, the locations 4604, etc.)may be provided as a ground truth input to a neural network, and theneural network may output predicted characteristics for twelve locations4608 on user interface 4600. During a fourth portion of the vision test,the predicted characteristics for the locations 4608 may be used to testthe user, and the feedback for the locations 4608 may be obtained. Suchfeedback for the locations 4608 along with the prior feedback (e.g., forone or more of the locations 4602, the locations 4604, the locations4606, etc.) may be provided as a ground truth input to a neural network,and the neural network may output one or more predicted characteristicsfor one or more further locations to be tested during a subsequentportion of the vision test. The neural network used for the third orfourth portion of the vision test may, for example, be the same instanceof the neural network from the first or second portion (or the other oneof the third or fourth portion) of the vision test or a differentinstance of the same neural network (or otherwise a different neuralnetwork) to accommodate or optimize for different input/outputconfigurations.

In a further use case, with respect to FIGS. 46E-46F, during a fifthportion of the vision test, four locations 4610 on user interface 4600(e.g., corresponding to a location in each near-peripheral quadrant of avisual field of a user) may be tested with one or more default startingcharacteristics, and feedback for the locations 4610 may be obtainedduring the fifth portion of the vision test. Such feedback for thelocations 4610 along with the prior feedback (e.g., for one or more ofthe locations 4602-4608) may be provided as a ground truth input to aneural network, and the neural network may output predictedcharacteristics for twelve locations 4612 on user interface 4600. Duringa sixth portion of the vision test, the predicted characteristics forthe locations 4612 may be used to test the user, and the feedback forthe locations 4612 may be obtained. Such feedback for the locations 4612along with the prior feedback (e.g., for one or more of the locations4602-4610) may be provided as a ground truth input to a neural network,and the neural network may output one or more predicted characteristicsfor one or more further locations to be tested during a subsequentportion of the vision test. The neural network used for the fifth orsixth portion of the vision test may, for example, be the same instanceof the neural network from the first, second, third, or fourth portion(or the other one of the fifth or sixth portion) of the vision test or adifferent instance of the same neural network (or otherwise a differentneural network) to accommodate or optimize for different input/outputconfigurations. In additional use cases, the foregoing operations may beperformed with respect to one or more other regions (e.g.,mid-peripheral regions, far-peripheral regions, or other defined regionsto be tested). It should be noted that, although the foregoing use caseswith respect to FIGS. 46A-46F begin with testing locations of thecentral region of the visual field and involve assessing regions inorder from their proximity to the center of the visual field, other usecases that begin testing locations in other regions (e.g.,near-peripheral, mid-peripheral, far-peripheral, etc.) or assess in adifferent order are contemplated.

In some embodiments, a set of predicted characteristics may be selectedto perform one or more portions of the vision test based on a confidencescore associated with the set of predicted characteristics. As anexample, the set of predicted characteristics may be selected over oneor more other sets of predicted characteristics (of the multiple sets)based on the confidence scores associated with the respective sets ofpredicted characteristics (e.g., selected for a given round or portionof the visual test presentation). In one use case, a pattern (e.g.,corresponding to the set of predicted characteristics) may be selectedbased on the confidence score associated with the pattern being greaterthan or equal to confidence scores associated with one or more otherpatterns (e.g., corresponding to the other sets of predictedcharacteristics). In another use case, the pattern may be selected basedon the confidence score associated with the pattern satisfying athreshold score (e.g., a predefined threshold score, a dynamic thresholdscore, etc.), and the other patterns may not be selected based on theconfidence scores associated with the other patterns failing to satisfythe threshold score.

As shown in FIG. 45, for example, patterns 4502 a-4502 f are examples ofcommon visual defect patterns, such as (a) tunnel vision defect withtemporal crescent sparing, (b) superior arcuate with peripheralbreakthrough and early inferior defect, (c) established superior arcuatedefect, (d) nasal step, (e) temporal wedge, and (f) superior,fixation-threatening paracentral defect. With respect to FIG. 45, thedark areas of patterns 4502 a-4502 f represent locations of a user'svisual field that have defects, and the white areas of patterns 4502a-4502 f represent locations of the user's visual field without visualdefects (e.g., as determined in accordance with a visual field testingstandard). It should be noted that, for convenience, other common visualdefect patterns (e.g., complete field loss, early superior paracentraldefect, etc.) or details regarding additional characteristics necessaryfor a user to see a stimulus at a visual defect location (along withfurther patterns) are not shown in FIG. 45 (e.g., details indicating acontrast level necessary for the user to see the stimulus), but arecontemplated. In one use case, a neural network (or one or moreinstances of the neural network) may be trained to predict one or morepatterns (e.g., where each pattern corresponds to a set of predictedcharacteristics for a set of user interface locations corresponding tolocations of the user's visual field). In a first portion of a visualtest presentation, an initial set of locations of a user interface maybe tested to determine threshold characteristics (e.g., minimum contrastlevels) under which the user is able to see stimuli presented atrespective locations of the initial set. Feedback (e.g., indicating suchthreshold characteristics) from the first portion of the visual testpresentation may be provided to the neural network, and the neuralnetwork may output a first pattern of predicted characteristics (e.g.,for all locations of the test set). As an example, the first pattern maybe outputted by the neural network based on the first pattern beingassociated with a confidence score satisfying a threshold score, theconfidence score being greater than or equal to confidence scores forother patterns predicted by the neural network, etc. During a secondportion of the visual test presentation, one or more predictedcharacteristics for locations of the outputted first pattern may then beused to test the user (e.g., via one or more techniques describedherein).

In some use cases, only a subset of the locations of the outputtedpattern are selected to be tested during the second portion of thevisual test presentation, and the feedback (e.g., indicating thresholdcharacteristics under which the user is able to see stimulus presentedat the respective selected locations) from the second portion of thevisual test presentation may be provided to a neural network (e.g., thesame or different instance of the neural network from the first portionof the visual test presentation, a different neural network, etc.), andthe neural network may output a second pattern of predictedcharacteristics (e.g., for all locations of the test set). One or morepredicted characteristics for locations of the outputted second patternmay then be used to test the user during a third portion of the visualtest presentation. The foregoing operations may be repeated forsubsequent portions of the visual test presentation until a thresholdnumber of locations have been tested or all locations of a test set havebeen tested.

In one use case, with respect to FIGS. 46A-46B, during a first portionof a vision test, four locations 4602 on user interface 4600 (e.g.,corresponding to a location in each central quadrant of a visual fieldof a user) may be initially tested (e.g., without prior knowledge) withone or more default starting characteristics, and feedback for thelocations 4602 (e.g., indicating threshold characteristics under whichthe user is able to see stimuli at the respective locations 4602) may beobtained during the first portion of the vision test. Such feedback maybe provided as a ground truth input to a neural network, and the neuralnetwork may output a first pattern of predicted characteristics (e.g.,for all locations of the test set). During a second portion of thevision test, predicted characteristics for the twelve locations 4604 ofthe outputted first pattern (e.g., a subset of the locations of thefirst pattern) may then be used to test the user. The feedback for thelocations 4604 (e.g., indicating the respective thresholdcharacteristics from the second portion of the vision test) may beobtained during the second portion of the vision test. Such feedback maybe provided to a neural network, and the neural network may output asecond pattern of predicted characteristics to be used to test the userduring a subsequent portion of the vision test. The neural network usedfor the second portion of the vision test may, for example, be the sameinstance of the neural network from the first portion of the vision testor a different instance of the same neural network (or otherwise adifferent neural network) to accommodate or optimize for differentinput/output configurations.

In another use case, with respect to FIGS. 46C-46D, during a thirdportion of the vision test, four locations 4606 on user interface 4600(e.g., corresponding to a location in each paracentral quadrant of avisual field of a user) may be tested with one or more default startingcharacteristics, and feedback for the locations 4606 may be obtained.Such feedback for the locations 4606 along with the prior feedback(e.g., for one or more of the locations 4602, the locations 4604, etc.)may be provided as a ground truth input to a neural network, and theneural network may output the second pattern of predictedcharacteristics (e.g., for all locations of the test set). During afourth portion of the vision test, predicted characteristics for thetwelve locations 4608 of the outputted second pattern (e.g., a subset ofthe locations of the second pattern) may be used to test the user, andthe feedback for the locations 4608 may be obtained. Such feedback forthe locations 4608 along with the prior feedback (e.g., for one or moreof the locations 4602, the locations 4604, the locations 4606, etc.) maybe provided as a ground truth input to a neural network, and the neuralnetwork may output a third pattern of predicted characteristics to beused to test the user during a subsequent portion of the vision test.The neural network used for the third or fourth portion of the visiontest may, for example, be the same instance of the neural network fromthe first or second portion (or the other one of the third or fourthportion) of the vision test or a different instance of the same neuralnetwork (or otherwise a different neural network) to accommodate oroptimize for different input/output configurations.

In a further use case, with respect to FIGS. 46E-46F, during a fifthportion of the vision test, four locations 4610 on user interface 4600(e.g., corresponding to a location in each near-peripheral quadrant of avisual field of a user) may be tested with one or more default startingcharacteristics, and feedback for the locations 4610 may be obtainedduring the fifth portion of the vision test. Such feedback for thelocations 4610 along with the prior feedback (e.g., for one or more ofthe locations 4602-4608) may be provided as a ground truth input to aneural network, and the neural network may output the third pattern ofpredicted characteristics (e.g., for all locations of the test set).During a sixth portion of the vision test, predicted characteristics forthe locations 4612 of the outputted third pattern (e.g., a subset of thelocations of the third pattern) may be used to test the user, and thefeedback for the locations 4612 may be obtained. Such feedback for thelocations 4612 along with the prior feedback (e.g., for one or more ofthe locations 4602-4610) may be provided as a ground truth input to aneural network, and the neural network may output a fourth pattern ofpredicted characteristics to be used to test the user during asubsequent portion of the vision test. The neural network used for thefifth or sixth portion of the vision test may, for example, be the sameinstance of the neural network from the first, second, third, or fourthportion (or the other one of the fifth or sixth portion) of the visiontest or a different instance of the same neural network (or otherwise adifferent neural network) to accommodate or optimize for differentinput/output configurations. In additional use cases, the foregoingoperations may be performed with respect to one or more other regions(e.g., mid-peripheral regions, far-peripheral regions, or other definedregions to be tested). It should be noted that, although the foregoinguse cases with respect to FIGS. 46A-46F begin with testing locations ofthe central region of the visual field and involve assessing regions inorder from their proximity to the center of the visual field, other usecases that begin testing locations in other regions (e.g.,near-peripheral, mid-peripheral, far-peripheral, etc.) or assess in adifferent order are contemplated.

In some embodiments, a set of predicted characteristics (for a set oflocations of a user interface) and a set of confidence scores associatedwith the set of locations may be obtained via a prediction model (e.g.,one or more instances of the prediction model). Based on the set ofconfidence scores, one or more locations of the set of locations may beselected to be tested during a visual test presentation. As an example,the locations may be selected over one or more other locations of theset of locations based on the set of confidence scores (e.g., selectedfor a given round or portion of the visual test presentation). In oneuse case, the locations may be selected based on confidence scoresassociated with the locations being greater than or equal to confidencescores associated with the other locations. In another use case, thelocations may be selected based on confidence scores associated with thelocations satisfying a threshold score (e.g., a predefined thresholdscore, a dynamic threshold score, etc.), and the other locations may notbe selected based on confidence scores associated with the locationsfailing to satisfy the threshold score. Additionally, or alternatively,the locations may be selected (e.g., over the other locations) based onan amount of locations to be tested (e.g., a fixed number or percentageof locations outputted by the prediction model that are to be testedduring a given round or portion of the visual test presentation), abounding box or other structure defining a region of a user's visualfield to be tested, or other criteria.

After the foregoing locations are selected, one or more stimuli may thenbe presented at the selected locations during the visual testpresentation based on one or more predicted characteristics associatedwith the selected locations. As an example, the predictedcharacteristics may include predictions of a threshold characteristicunder which a user (taking the visual test) will see a stimulus at therespective selected locations (e.g., one or more predicted brightnesslevels, contrast levels, saturation levels, sharpness levels, etc.). Afirst stimulus may, for example, initially be presented at a firstlocation of the locations based on a first predicted characteristicassociated with the first location. In one use case, the first stimulusmay initially be presented at the first location under the firstpredicted characteristic (e.g., a first contrast level under which theuser is predicted to see the first stimulus). In another use case, thefirst predicted characteristic may be used to determine at least onecharacteristic that is adjacent to the first predicted characteristic ina range of characteristics, and the first stimulus may be initiallypresented at the first location under the adjacent characteristic.

In some embodiments, feedback indicating one or more thresholdcharacteristics (under which a user sees one or more stimuli presentedon a user interface) may be provided to a prediction model. Based on thefeedback, a set of predicted characteristics (for a set of locations ofthe user interface) and a set of confidence scores associated with theset of locations may be obtained via the prediction model. As discussedabove, locations of the set of locations may be selected based on theconfidence scores, and stimuli may be presented at the selectedlocations based on predicted characteristics associated with theselected locations. From such stimuli presentation, additional feedbackindicating threshold characteristics (under which the user sees thestimuli at the selected locations, respectively) may be obtained. Basedon the foregoing feedback(s), visual defect information may be generatedfor the user (e.g., in accordance with one or more techniques describedherein).

In some embodiments, a ground truth input for a prediction model may beinitialized, and a set of predicted characteristics (for a set oflocations of the user interface) and a set of confidence scoresassociated with the set of locations may be obtained via the predictionmodel based on the ground truth input. As an example, the ground truthinput may include initial feedback related to an initial set of stimulipresented at an initial set of locations of the user interface under arange of characteristics (e.g., initial feedback indicating thresholdcharacteristics of the range of characteristics under which a user seeseach stimulus of the initial set of stimuli). As discussed above,locations of the set of locations may be selected based on theconfidence scores (e.g., selected for a given round or portion of thevisual test presentation), and stimuli may be presented at the selectedlocations (e.g., during the given round/portion) based on predictedcharacteristics associated with the selected locations. From suchstimuli presentation (e.g., during the given round/portion), additionalfeedback indicating threshold characteristics (under which the user seesthe stimuli at the selected locations, respectively) may be obtained.Based on the additional feedback, the ground truth input may be updatedfor a subsequent round or portion of the visual test presentation. As anexample, the ground truth input may be updated to include the initialfeedback, the additional feedback, or other input information.

In one use case, with respect to FIGS. 47A-47B, during a first portionof a vision test, four locations 4702 on user interface 4700 (e.g.,corresponding to a location in each central quadrant of a visual fieldof a user) may be initially tested (e.g., without prior knowledge) withone or more default starting characteristics, and feedback for thelocations 4702 (e.g., indicating threshold characteristics under whichthe user is able to see stimuli at the respective locations 4702) may beobtained during the first portion of the vision test. Such feedback maybe provided as a ground truth input to a neural network, and a first setof predicted characteristics for a first set of locations 4704 (e.g., atleast part of a first pattern of predicted characteristics) and a firstset of confidence scores associated with the first set of locations 4704may be obtained via the neural network (e.g., via one or more hidden oroutput layers of the neural network). Locations 4704 a may be selectedto be tested during a second portion of the vision test over locations4704 b based on the predictions for locations 4704 a being highconfidence predictions and the predictions for locations 4704 b beinglow confidence predictions. As such, for example, the predictedcharacteristics for the locations 4704 a may be used to test the userduring the second portion of the vision test, while the locations 4704 bare not tested during the second portion of the vision test. Thefeedback for the locations 4704 a (e.g., indicating the respectivethreshold characteristics from the second portion of the vision test)may be obtained during the second portion of the vision test, and suchfeedback may be provided as a ground truth input to a neural network toobtain one or more predicted characteristics for one or more furtherlocations to be tested during a subsequent portion of the vision test.The neural network used for the second portion of the vision test may,for example, be the same instance of the neural network from the firstportion of the vision test or a different instance of the same neuralnetwork (or otherwise a different neural network) to accommodate oroptimize for different input/output configurations.

In another use case, with respect to FIGS. 47C-47D, during a thirdportion of the vision test, the feedback for the locations 4704 a alongwith the prior feedback (e.g., for one or more of the locations 4702)may be provided as a ground truth input to a neural network, and asecond set of predicted characteristics for a second set of locations4706 (e.g., at least part of a second pattern of predictedcharacteristics) and a second set of confidence scores associated withthe second set of locations 4706 may be obtained via the neural network(e.g., via one or more hidden or output layers of the neural network).Locations 4706 a (e.g., which includes location 4704 b) may be selectedto be tested during a fourth portion of the vision test over locations4706 b based on the predictions for locations 4706 a being highconfidence predictions and the predictions for locations 4706 b beinglow confidence predictions. As such, for example, the predictedcharacteristics for the locations 4706 a may be used to test the userduring the fourth portion of the vision test, while the locations 4706 bare not tested during the fourth portion of the vision test. Thefeedback for the locations 4706 a may be obtained during the fourthportion of the vision test, and such feedback may be provided as aground truth input to a neural network to obtain one or more predictedcharacteristics for one or more further locations to be tested during asubsequent portion of the vision test. The neural network used for thethird or fourth portion of the vision test may, for example, be the sameinstance of the neural network from the first or second portion (or theother one of the third or fourth portion) of the vision test or adifferent instance of the same neural network (or otherwise a differentneural network) to accommodate or optimize for different input/outputconfigurations.

In a further use case, with respect to FIGS. 47E-47F, during a fifthportion of the vision test, the feedback for the locations 4706 a alongwith the prior feedback (e.g., for one or more of the locations 4702 and4704 a) may be provided as a ground truth input to a neural network, anda third set of predicted characteristics for a third set of locations4708 (e.g., at least part of a third pattern of predictedcharacteristics) and a third set of confidence scores associated withthe third set of locations 4708 may be obtained via the neural network(e.g., via one or more hidden or output layers of the neural network).Locations 4708 a (e.g., which includes location 4706 b) may be selectedto be tested during a sixth portion of the vision test over locations4708 b based on the predictions for locations 4708 a being highconfidence predictions and the predictions for locations 4708 b beinglow confidence predictions. As such, for example, the predictedcharacteristics for the locations 4708 a may be used to test the userduring the sixth portion of the vision test, while the locations 4708 bare not tested during the sixth portion of the vision test. The feedbackfor the locations 4708 a may be obtained during the sixth portion of thevision test, and such feedback may be provided as a ground truth inputto a neural network to obtain one or more predicted characteristics forone or more further locations to be tested during a subsequent portionof the vision test. In additional use cases, the foregoing operationsmay be performed with respect to one or more other regions (e.g.,mid-peripheral regions, far-peripheral regions, or other defined regionsto be tested). It should be noted that, although the foregoing use caseswith respect to FIGS. 47A-47F begin with testing locations of thecentral region of the visual field and involve assessing regions inorder from their proximity to the center of the visual field, other usecases that begin testing locations in other regions (e.g.,near-peripheral, mid-peripheral, far-peripheral, etc.) or assess in adifferent order are contemplated.

FIGS. 41-43 and 48-49 are example flowcharts of processing operations ofmethods that enable the various features and functionality of the systemas described in detail above. The processing operations of each methodpresented below are intended to be illustrative and non-limiting. Insome embodiments, for example, the methods may be accomplished with oneor more additional operations not described, and/or without one or moreof the operations discussed. Additionally, the order in which theprocessing operations of the methods are illustrated (and describedbelow) is not intended to be limiting.

In some embodiments, the methods may be implemented in one or moreprocessing devices (e.g., a digital processor, an analog processor, adigital circuit designed to process information, an analog circuitdesigned to process information, a state machine, and/or othermechanisms for electronically processing information). The processingdevices may include one or more devices executing some or all of theoperations of the methods in response to instructions storedelectronically on an electronic storage medium. The processing devicesmay include one or more devices configured through hardware, firmware,and/or software to be specifically designed for execution of one or moreof the operations of the methods.

FIG. 41 shows a flowchart of a method 4100 of facilitating modificationrelated to a vision of a user via a prediction model, in accordance withone or more embodiments.

In an operation 4102, a visual test presentation may be provided to auser. As an example, the visual test presentation may include a set ofstimuli. The set of stimuli may include light stimuli, text, or imagesdisplayed to the user. Operation 4102 may be performed by a subsystemthat is the same as or similar to testing subsystem 122, in accordancewith one or more embodiments.

In an operation 4104, one or more characteristics of one or more eyes ofthe user may be monitored. As an example, the eye characteristics may bemonitored during the visual test presentation. The eye characteristicsmay include gaze direction, pupil size, limbus position, visual axis,optical axis, or other characteristics (e.g., during the visual testpresentation). Operation 4104 may be performed by a subsystem that isthe same as or similar to testing subsystem 122, in accordance with oneor more embodiments.

In an operation 4106, feedback related to the set of stimuli may beobtained. As an example, the feedback may be obtained during the visualtest presentation, and the feedback may indicate whether or how the usersees one or more stimuli of the set. Additionally, or alternatively, thefeedback may include one or more characteristics related to the eyesoccurring when the stimuli are displayed. Operation 4106 may beperformed by a subsystem that is the same as or similar to testingsubsystem 122, in accordance with one or more embodiments.

In an operation 4108, the feedback related to the set of stimuli may beprovided to a prediction model. As an example, the feedback may beprovided to the prediction model during the visual test presentation,and the prediction model may be configured based on the feedback and theeye characteristic information. As another example, based on thefeedback, the prediction model may be configured to provide modificationparameters or functions to be applied to image data (e.g., live videostream) to generate an enhanced presentation related to the image data.Operation 4108 may be performed by a subsystem that is the same as orsimilar to testing subsystem 122, in accordance with one or moreembodiments.

In an operation 4110, video stream data and the user's current eyecharacteristics information (e.g., indicating the user's current eyecharacteristics) may be provided to the prediction model. As an example,the video stream data may be a live video stream obtained via one ormore cameras of a wearable device of the user, and the live video streamand the current eye characteristics information may be provided to theprediction model in real-time. Operation 4110 may be performed by asubsystem that is the same as or similar to visioning subsystem 124, inaccordance with one or more embodiments.

In an operation 4112, a set of modification parameters or functions maybe obtained from the prediction model. As an example, the set ofmodification parameters or functions may be obtained from the predictionmodel based on the video stream and the current eye characteristicsinformation being provided to the prediction model. As another example,the set of modification parameters or functions may be configured to beapplied to the video stream to generate an enhanced image (e.g., thataccommodates for dynamic aberrations of the user). Additionally, oralternatively, the set of modification parameters or functions may beconfigured to be applied to dynamically adjust one or more displayportions of a display. Operation 4112 may be performed by a subsystemthat is the same as or similar to visioning subsystem 124, in accordancewith one or more embodiments.

In an operation 4114, an enhanced image may be caused to be displayed tothe user based on the video stream data and the set of modificationparameters or functions. Operation 4114 may be performed by a subsystemthat is the same as or similar to visioning subsystem 124, in accordancewith one or more embodiments.

FIG. 42 shows a flowchart of a method 4200 of facilitating an increasein a field of view of a user via combination of portions of multipleimages of a scene, in accordance with one or more embodiments.

In an operation 4202, a plurality of images of a scene may be obtained.As an example, the images may be obtained via one or more cameras (e.g.,of a wearable device) at different positions or orientations. Operation4202 may be performed by a subsystem that is the same as or similar tovisioning subsystem 124, in accordance with one or more embodiments.

In an operation 4204, a region common to the images may be determined.As an example, the common region may correspond to respective portionsof the images that have the same or similar characteristics as oneanother. Operation 4204 may be performed by a subsystem that is the sameas or similar to visioning subsystem 124, in accordance with one or moreembodiments.

In an operation 4206, for each image of the images, a region of theimage divergent from a corresponding region of at least another image(of the images) may be determined. As an example, each divergent regionmay correspond to a portion of one of the images that is distinct fromall the other corresponding portions of the other images. Operation 4206may be performed by a subsystem that is the same as or similar tovisioning subsystem 124, in accordance with one or more embodiments.

In an operation 4208, an enhanced image may be generated based on thecommon region and the divergent regions. As an example, the enhancedimage may be generated such that (i) a first region of the enhancedimage includes a representation of the common region and (ii) a secondregion of the enhanced image comprises representations of the divergentregions. As another example, the enhanced image may be generated suchthat the second region is around the first region in the enhanced image.Operation 4208 may be performed by a subsystem that is the same as orsimilar to visioning subsystem 124, in accordance with one or moreembodiments.

In an operation 4210, the enhanced image may be displayed. As anexample, the enhanced image may be displayed via one or more displays ofa wearable device of the user. Operation 4210 may be performed by asubsystem that is the same as or similar to visioning subsystem 124, inaccordance with one or more embodiments.

FIG. 43 shows a flowchart of a method 4300 of facilitating enhancementof a field of view of a user via one or more dynamic display portions onone or more transparent displays, in accordance with one or moreembodiments.

In an operation 4302, one or more changes related to one or more eyes ofa user may be monitored. As an example, the eye changes may include aneye movement, a change in gaze direction, a pupil size change, or otherchanges. Operation 4302 may be performed by a subsystem that is the sameas or similar to visioning subsystem 124, in accordance with one or moreembodiments.

In an operation 4304, an adjustment of one or more transparent displayportions of a wearable device may be caused based on the monitoredchanges. As an example, one or more positions, shapes, or sizes of thetransparent display portions of the wearable device may be adjustedbased on the monitored changes. Operation 4304 may be performed by asubsystem that is the same as or similar to visioning subsystem 124, inaccordance with one or more embodiments.

In an operation 4306, an enhanced image (e.g., derived from live imagedata) may be displayed on one or more other display portions of thewearable device. As an example, at least one of the other displayportions may be around at least one of the transparent display portionsof the wearable device such that the enhanced image is displayed aroundthe transparent display portion (e.g., and not within the transparentdisplay portions). Operation 4306 may be performed by a subsystem thatis the same as or similar to visioning subsystem 124, in accordance withone or more embodiments.

FIG. 48 shows a flowchart of a method 4800 of facilitating visiontesting via pattern-based selection of a testing location, in accordancewith one or more embodiments.

In an operation 4802, first feedback related to a first set of stimuli(presented at a first set of locations of a user interface) may beobtained. In some embodiments, the first set of stimuli may be presentedunder a range of characteristics. Operation 4802 may be performed by asubsystem that is the same as or similar to testing subsystem 122, inaccordance with one or more embodiments.

In an operation 4804, the first feedback related to the first set ofstimuli may be provided to a neural network. In some embodiments, theneural network may be configured based on the first feedback. Operation4802 may be performed by a subsystem that is the same as or similar totesting subsystem 122, in accordance with one or more embodiments.

In an operation 4806, a pattern for a second set of locations of theuser interface may be obtained. In some embodiments, the pattern mayindicate predicted characteristics for user interface locations of thesecond set of locations. In some embodiments, the second set oflocations may include a greater number of user interface locations thanthe first set of locations. Operation 4806 may be performed by asubsystem that is the same as or similar to testing subsystem 122, inaccordance with one or more embodiments.

In an operation 4808, a second set of stimuli may be presented at theuser interface location under at least one characteristic of the rangeof characteristics. For example, the second set of stimuli may bepresented based on the pattern indicating the predicted characteristics.In some embodiments, the second set of stimuli may be presented suchthat (i) a first stimulus is initially presented at a user interfacelocation of the second set of locations based on a first predictedcharacteristic indicated by the pattern and (ii) a second stimulus isinitially presented at another user interface location of the second setof locations based on a second predicted characteristic indicated by thepattern. Operation 4808 may be performed by a subsystem that is the sameas or similar to testing subsystem 122, in accordance with one or moreembodiments.

In an operation 4810, second feedback related to the second set ofstimuli may be obtained. For example, the second feedback may indicate athreshold characteristic of the range of characteristics under which theuser sees each stimulus of the second set of stimuli. Operation 4810 maybe performed by a subsystem that is the same as or similar to testingsubsystem 122, in accordance with one or more embodiments.

In an operation 4812, visual defect information may be generated for theuser. For example, the visual defect information may be generated basedon the first feedback related to the first set of stimuli and the secondfeedback related to the second set of stimuli. Operation 4812 may beperformed by a subsystem that is the same as or similar to testingsubsystem 122, in accordance with one or more embodiments.

FIG. 49 shows a flowchart of a method 4900 of facilitating visiontesting via confidence-based selection of a testing location subset, inaccordance with one or more embodiments.

In an operation 4902, first feedback indicating one or more thresholdcharacteristics (under which a user sees one or more stimuli presentedon a user interface) may be obtained. As an example, each such stimulusmay be presented to the user in a descending or ascending order ofcharacteristics (e.g., brightness levels, contrast levels, saturationlevels, sharpness levels, or another range of characteristics) at agiven user interface locations that correspond to a given visual fieldlocations of the user's field of view. A threshold characteristic may bea characteristic above which a user can see the stimulus and below whichthe user cannot see the stimulus (or vice versa). Operation 4902 may beperformed by a subsystem that is the same as or similar to testingsubsystem 122, in accordance with one or more embodiments.

In an operation 4904, a set of predicted characteristics for a set oflocations of the user interface and a set of confidence scoresassociated with the set of locations may be obtained via a predictionmodel based on the first feedback. As an example, the first feedback maybe provided as input to the prediction model, and the prediction modelmay output (i) a first predicted characteristic for a first location anda first confidence score associated with the first location and itsprediction, (ii) a second predicted characteristic for a second locationand a second confidence score associated with the second location andits prediction, (iii) a third predicted characteristic for a thirdlocation and a third confidence score associated with the third locationand its prediction, and (iv) so on. As an example, such confidencescores may be probabilities related to the accuracy of the predictionsof threshold characteristics under which the user is able to see astimulus at the respective locations (e.g., user interface locationscorresponding to locations of the user's visual field). Operation 4904may be performed by a subsystem that is the same as or similar totesting subsystem 122, in accordance with one or more embodiments.

In an operation 4906, one or more locations of the set of locations(that are to be tested during a visual test presentation) may beselected based on the set of confidence scores. As an example, thelocations may be selected over one or more other locations of the set oflocations based on the set of confidence scores (e.g., selected for agiven round or portion of the visual test presentation). In one usecase, the locations may be selected based on (i) confidence scoresassociated with the locations being greater than or equal to confidencescores associated with the other locations, (ii) the confidence scoresassociated with the locations satisfying a threshold score (e.g., apredefined threshold score, a dynamic threshold score, etc.), (iii) anamount of locations to be tested (e.g., a fixed number or percentage oflocations outputted by the prediction model that are to be tested duringa given round or portion of the visual test presentation), (iv) abounding box or other structure defining a region of a user's visualfield to be tested, or (v) other criteria. Operation 4906 may beperformed by a subsystem that is the same as or similar to testingsubsystem 122, in accordance with one or more embodiments.

In an operation 4908, one or more stimuli may be caused to be presentedat the selected locations during the visual test presentation based onone or more predicted characteristics associated with the selectedlocations. As an example, the predicted characteristics may includepredictions of a threshold characteristic under which a user (taking thevisual test) will see a stimulus at the respective selected locations(e.g., one or more predicted brightness levels, contrast levels,saturation levels, sharpness levels, etc.). In one use case, a firststimulus may initially be presented at the first location of thelocations under a first predicted contrast level associated with thefirst location. If it is determined that the user sees the firststimulus under the first predicted contrast level, such determinationmay be stored as feedback indicating the first predicted contrast levelas a threshold contrast level for the first location. On the other hand,if it is determined that the user cannot see the first stimulus underthe first predicted contrast level, the contrast level may be increasedto the next contrast level. Such process may be repeated for the firstlocation until the user sees the first stimulus or a maximum contrastlevel is reached (e.g., the first stimulus is presented at the highestcontrast level of a set of available contrast levels). Operation 4908may be performed by a subsystem that is the same as or similar totesting subsystem 122, in accordance with one or more embodiments.

In an operation 4910, second feedback indicating one or more thresholdcharacteristics (under which the user sees the stimuli at the selectedlocations) may be obtained. As indicated above, if it is determined thatthe user sees a given stimulus at a selected location under a predictedcontrast level, such determination may be stored as part of the secondfeedback (e.g., indicating that the predicted contrast level is athreshold contrast level for the user to see the stimulus at theselected location). Such feedback may also indicate one or more selectedlocations for which it is determined that the user was not able to seestimuli (e.g., the user was not able to see stimuli at those selectedlocations when the stimuli were presented at the highest contrast levelof a set of available contrast levels). Operation 4910 may be performedby a subsystem that is the same as or similar to testing subsystem 122,in accordance with one or more embodiments.

In an operation 4912, visual defect information may be generated for theuser based on the first feedback and the second feedback. As an example,the visual defect information may include information indicatingdefective visual field portions, such as (i) locations of the user'svisual field where the user is determined to have no vision (e.g., theuser did not see stimuli presented at the highest contrast level), (ii)locations of the user's visual field where the user is determined tohave reduced vision (e.g., the user was able to see stimuli atrespective contrast levels greater than a particular threshold contrastlevel), (iii) the amount of vision reduction or sensitivities for eachof such locations where the user is determined to have reduced vision,(iv) etc. Operation 4912 may be performed by a subsystem that is thesame as or similar to testing subsystem 122, in accordance with one ormore embodiments.

In some embodiments, the various computers and subsystems illustrated inFIG. 1A may include one or more computing devices that are programmed toperform the functions described herein. The computing devices mayinclude one or more electronic storages (e.g., prediction database(s)132, which may include training data database(s) 134, model database(s)136, etc., or other electric storages), one or more physical processorsprogrammed with one or more computer program instructions, and/or othercomponents. The computing devices may include communication lines orports to enable the exchange of information with a network (e.g.,network 150) or other computing platforms via wired or wirelesstechniques (e.g., Ethernet, fiber optics, coaxial cable, WiFi,Bluetooth, near field communication, or other technologies). Thecomputing devices may include a plurality of hardware, software, and/orfirmware components operating together. For example, the computingdevices may be implemented by a cloud of computing platforms operatingtogether as the computing devices.

The electronic storages may include non-transitory storage media thatelectronically stores information. The electronic storage media of theelectronic storages may include one or both of (i) system storage thatis provided integrally (e.g., substantially non-removable) with serversor client devices or (ii) removable storage that is removablyconnectable to the servers or client devices via, for example, a port(e.g., a USB port, a firewire port, etc.) or a drive (e.g., a diskdrive, etc.). The electronic storages may include one or more ofoptically readable storage media (e.g., optical disks, etc.),magnetically readable storage media (e.g., magnetic tape, magnetic harddrive, floppy drive, etc.), electrical charge-based storage media (e.g.,EEPROM, RAM, etc.), solid-state storage media (e.g., flash drive, etc.),and/or other electronically readable storage media. The electronicstorages may include one or more virtual storage resources (e.g., cloudstorage, a virtual private network, and/or other virtual storageresources). The electronic storage may store software algorithms,information determined by the processors, information obtained fromservers, information obtained from client devices, or other informationthat enables the functionality as described herein.

The processors may be programmed to provide information processingcapabilities in the computing devices. As such, the processors mayinclude one or more of a digital processor, an analog processor, adigital circuit designed to process information, an analog circuitdesigned to process information, a state machine, and/or othermechanisms for electronically processing information. In someembodiments, the processors may include a plurality of processing units.These processing units may be physically located within the same device,or the processors may represent processing functionality of a pluralityof devices operating in coordination. The processors may be programmedto execute computer program instructions to perform functions describedherein of subsystems 112-124 or other subsystems. The processors may beprogrammed to execute computer program instructions by software;hardware; firmware; some combination of software, hardware, or firmware;and/or other mechanisms for configuring processing capabilities on theprocessors.

It should be appreciated that the description of the functionalityprovided by the different subsystems 112-124 described herein is forillustrative purposes, and is not intended to be limiting, as any ofsubsystems 112-124 may provide more or less functionality than isdescribed. For example, one or more of subsystems 112-124 may beeliminated, and some or all of its functionality may be provided byother ones of subsystems 112-124. As another example, additionalsubsystems may be programmed to perform some or all of the functionalityattributed herein to one of subsystems 112-124.

The present techniques may be used in any number of applications,including for example for otherwise healthy subjects frequently affectedby quick onset of optical pathologies, subjects such as soldiers andveterans. Loss of visual field compromises the ability of soldiers,veterans, other affected patients to perform their essential tasks aswell as daily life activities. This visual disability compromises theirindependence, safety, productivity and quality of life and leads to lowself-esteem and depression. Despite recent scientific advances,treatment options to reverse existing damage of the retina, optic nerveor visual cortex are limited. Thus, treatment relies on offeringpatients with visual aids to maximize their functionality. Currentvisual aids fall short in achieving those goals. This underlines theneed for having better visual aids to improve visual performance,quality of life and safety. The techniques herein, integrated intospectacles device, are able to diagnose and mitigate common quick onseteye injuries, such as military-related eye injuries and diseases, thatcause visual field defects, in austere or remote, as well as general,environments. The techniques herein are able to diagnose and quantifyvisual field defects. Using this data, the devices process, inreal-time, patients' field of view and fits and projects correctedimages on their remaining functional visual field. Thus, minimizing thenegative effect of the blind (or reduced) part of visual field onpatients' visual performance. Moreover, the fact that the spectaclesdevice does not rely on another clinical device to diagnose visual fielddefects make them specifically useful in austere and remoteenvironments. Similarly, the present techniques may be used to augmentthe visual field of normal subjects to have a better than normal visualfield or vision.

Although the present invention has been described in detail for thepurpose of illustration based on what is currently considered to be themost practical and preferred embodiments, it is to be understood thatsuch detail is solely for that purpose and that the invention is notlimited to the disclosed embodiments, but, on the contrary, is intendedto cover modifications and equivalent arrangements that are within thescope of the appended claims. For example, it is to be understood thatthe present invention contemplates that, to the extent possible, one ormore features of any embodiment can be combined with one or morefeatures of any other embodiment.

The present techniques will be better understood with reference to thefollowing enumerated embodiments:

A1. A method comprising: providing a presentation (e.g., a visual testpresentation or other presentation) comprising a set of stimuli to auser; obtaining feedback related to the set of stimuli (e.g., thefeedback indicating whether or how the user senses one or more stimuliof the set); providing the feedback related to the set of stimuli to amodel (e.g., a machine learning model or other model), the model beingconfigured based on the feedback related to the set of stimuli.A2. The method of embodiment A1, further comprising: providing liveimage data, eye characteristic information, or environmentcharacteristic information to the model to obtain an enhanced imagederived from the live image data; and causing an enhanced image to bedisplayed to the user, the eye characteristic information indicating oneor more characteristics of one or more eyes of the user that occurredduring a live capture of the live image data, the environmentcharacteristic information indicating one or more characteristics of theenvironment that occurred during the live capture of the live imagedata.A3. The method of embodiment A2, further comprising: obtaining theenhanced image from the model based on the live image data, eyecharacteristic information, or environment characteristic informationbeing provided to the model.A4. The method of embodiment A2, further comprising: obtaining one ormore modification parameters from the model based on the live imagedata, eye characteristic information, or environment characteristicinformation being provided to the model; and generating the enhancedimage based on the live image data or the one or more modificationparameters to obtain the enhanced image.A5. The method of embodiment A4, wherein the one or more modificationparameters comprises one or more transformation parameters, brightnessparameters, contrast parameters, saturation parameters, or sharpnessparameters.A6. The method of any of embodiments A1-A5, wherein obtaining thefeedback related to the set of stimuli comprises obtaining an eye imagecaptured during the presentation, the eye image being an image of an eyeof the user, and wherein providing the feedback related to the set ofstimuli comprises providing the eye image to the model.A7. The method of any of embodiment A5, wherein the eye image is anocular image, an image of a retina of the eye, or an image of a corneaof the eye.A8. The method of any of embodiments A1-A7, wherein obtaining thefeedback related to the set of stimuli comprises obtaining an indicationof a response of the user to one or more stimuli of the set of stimulior an indication of a lack of response of the user to one or morestimuli of the set of stimuli, and wherein providing the feedbackrelated to the set of stimuli comprises providing the indication of theresponse or the indication of the lack of response to the model.A9. The method of embodiment A8, wherein the response comprises an eyemovement, a gaze direction, a pupil size change, or a user modificationof one or more stimuli via user input of the user.A10. The method of embodiment A9, wherein the user modificationcomprises a movement of one or more stimuli via user input of the useror supplemental data provided via user input of the user over one ormore stimuli displayed to the user.A11. The method of any of embodiments A1-A10, further comprising:obtaining a second set of stimuli, the second set of stimuli beinggenerated based on the model's processing of the set of stimuli and thefeedback related to the set of stimuli; causing the second set ofstimuli to be displayed to the user; obtaining feedback related to thesecond set of stimuli (e.g., the feedback indicating whether or how theuser sees one or more stimuli of the second set); and providing thefeedback related to the second set of stimuli to the model, the modelbeing further configured based on the feedback related to the second setof stimuli.A12. The method of any of embodiments A1-A11, further comprising:determining, via the model, a defective visual field portion of a visualfield of the user based on the feedback related to the set of stimuli,the visual field of the user comprising visual field portions, thedefective visual field portion being one of the visual field portionsthat fails to satisfy one or more vision criteria.A13. The method of embodiment A12, wherein the enhanced image is basedon one or more transformations corresponding to the defective visualfield portion of the live image data such that an image portion of thelive image data is represented in an image portion of the enhanced imageoutside of the defective visual field portion.A14. The method of any of embodiments A12-A13, wherein the enhancedimage is based on one or more brightness or contrast modifications ofthe live image data such that (i) a brightness, contrast, or sharpnesslevel increase is applied to an image portion of the live image datacorresponding to the defective visual field portion to generate acorresponding image portion of the enhanced image and (ii) thebrightness, contrast, or sharpness level increase is not applied toanother image portion of the live stream data to generate acorresponding image portion of the enhanced image.A15. The method of any of embodiments A12-A14, further comprising:detecting an object (e.g., in the defective visual field portion orpredicted to be in the defective visual field portion); determining thatthe object is not sufficiently in any image portion of the enhancedimage that corresponds to at least one of the visual field portionssatisfying the one or more vision criteria; generating a predictionindicating that the object will come in physical contact with the user;and causing an alert to be displayed (e.g., over the enhanced image)based on (i) the prediction of physical contact and (ii) thedetermination that the object is not sufficiently any image portion ofthe enhanced image that corresponds to at least one of the visual fieldportions satisfying the one or more vision criteria, wherein the alertindicates an oncoming direction of the object.A16. The method of any of embodiments A1-15, wherein one or more of theforegoing operations are performed by a wearable device.A17. The method of embodiment A16, wherein the wearable device comprisesone or more cameras configured to capture the live image data and one ormore display portions configured to display one or more enhanced images.A18. The method of any of embodiments A16-A17, wherein the one or moredisplay portions comprise first and second display portions of thewearable device.A19. The method of embodiment A18, wherein the wearable device comprisesa first monitor comprising the first display portion and a secondmonitor comprising the second display portion.A20. The method of any of embodiments A16-A19, wherein the one or moredisplay portions comprise one or more dynamic display portions on one ormore transparent displays of the wearable device, and wherein one ormore enhanced images are displayed on the one or more display portions.A21. The method of any of embodiments A1-A20, further comprising:monitoring one or more changes related to one or more eyes of the user.A22. The method of embodiment 21, further comprising: providing the oneor more changes as further feedback to the model; and obtaining one ormore modification parameters from the model based on the live imagedata, eye characteristic information, or environment characteristicinformation being provided to the model; and generating the enhancedimage based on the live image data and the one or more modificationparameters to obtain the enhanced image.A23. The method of any of embodiments A21-A22, further comprising:causing, based on the monitoring, an adjustment of one or morepositions, shapes, sizes, or transparencies of the first or seconddisplay portions on one or more transparent displays of the wearabledevice, wherein causing the enhanced image to be displayed comprisescausing the enhanced image to be displayed on the first or seconddisplay portions.A24. The method of any of embodiments A1-A23, wherein the modelcomprises a neural network or other machine learning model.A25. The method of any of embodiments A1-A24, wherein the presentationcomprises presenting each stimulus of the set of stimuli under a set ofcharacteristics (e.g., under a range of characteristics in ascending ordescending order).A26. The method of any of embodiments A1-A25, wherein the feedbackrelated to the set of stimuli comprises a threshold characteristic ofthe set of characteristics under which the user sees each stimulus ofthe set of stimuli.A27. The method of any of embodiments A1-A26, further comprising:subsequent to the configuration of the model, obtaining, via the model,a predicted characteristic with respect to a user interface location ofa user interface; presenting, based on the predicted characteristic, astimulus at the user interface location under at least onecharacteristic of the set of characteristics; obtaining second feedbackrelated to the stimulus, the second feedback related to the stimulusindicating a threshold characteristic of the set of characteristicsunder which the user sees the stimulus; and generating visual defectinformation for the user based on the first feedback related to the setof stimuli and the second feedback related to the stimulus.A28. The method of embodiment A27, wherein the predicted characteristiccomprises a prediction of the threshold characteristic of the set ofcharacteristics under which the user sees the stimulus.A29. The method of any of embodiments A27-A28, wherein the stimulus isinitially presented at the user interface location under the predictedcharacteristic.A30. The method of embodiment A29, wherein the stimulus is initiallypresented at the user interface location under at least onecharacteristic that is adjacent, in the set of characteristics, to thepredicted characteristic.A31. The method of any of embodiments A27-A30, wherein the one or moreuser interface locations of the user interface correspond to visualfield locations.A32. The method of embodiment A31, wherein the visual defect informationdescribes one or more visual field locations at which the user hasvisual defects.A33. The method of embodiment A32, wherein the one or more visual fieldlocations at which the user has visual defects correspond to one or moreuser interface locations at which one or more threshold characteristicsunder which the user sees corresponding stimuli breach a predeterminedlevel.A34. The method of any of embodiments A27-A33, wherein the set ofcharacteristics comprise contrast levels, saturation levels, orsharpness levels.B1. A method comprising: obtaining a plurality of images of a scene;determining a region common to the images; for each image of the images,determining a region of the image divergent from a corresponding regionof at least another image of the images; generating an enhanced imagebased on the common region and the divergent regions; and causing theenhanced image to be displayed.B2. The method of embodiment B1, wherein generating the enhanced imagecomprises generating the enhanced image based on the common region andthe divergent regions such that (i) a first region of the enhanced imagecomprises a representation of the common region (ii) a second region ofthe enhanced image comprises representations of the divergent regions,and (iii) the second region is around the first region in the enhancedimage.B3. The method of embodiment B2, wherein generating the enhanced imagecomprises generating the enhanced image based on the common region, thedivergent regions, and a second region common to the images such that(i) the first region of the enhanced image comprises the representationof the common region and a representation of the second common regionand (ii) the second region of the enhanced image comprisesrepresentations of the divergent regions.B4. The method of any of embodiments B1-B3, wherein the common region isa region of at least one of the images that corresponds to a macularregion of a visual field of an eye or to a region within the macularregion of the visual field.B5. The method of any of embodiments B1-B4, wherein each of thedivergent regions is a region of at least one of the images thatcorresponds to a peripheral region of a visual field of an eye or to aregion within the peripheral region of the visual field.B6. The method of any of embodiments B1-B5, further comprising:performing shifting of each image of the images, wherein generating theenhanced image comprises generating the enhanced image based on thecommon region and the divergent regions subsequent to the performance ofthe shifting.B7. The method of embodiment B6, wherein performing the shiftingcomprises performing shifting of each image of the images such that asize of the common region is decreased and a size of at least one of thedivergent regions is increased.B8. The method of any of embodiments B1-B7, further comprising:performing resizing of one or more regions of the images, whereingenerating the enhanced image comprises generating the enhanced imagebased on the common region and the divergent regions subsequent to theperformance of the resizing.B9. The method of embodiment B8, wherein performing the resizingcomprises performing resizing of one or more regions of the images suchthat an extent of any resizing of the common region is different than anextent of any resizing of at least one of the divergent regions.B10. The method of any of embodiments B8-B9, wherein performing theresizing comprises performing the resizing of one or more regions of theimages such that a percentage change in size of the common regionrepresented in the first region of the enhanced image is greater than orless than a percentage change in size of at least one of the divergentregions represented in the second region of the enhanced image.B11. The method of embodiment B10, wherein the percentage change in sizeof at least one of the divergent regions is zero, and wherein thepercentage change in size of the common region is greater than zero.B12. The method of embodiment B10, wherein the percentage change in sizeof at least one of the divergent regions is greater than zero, andwherein the percentage change in size of the common region is zero.B13. The method of any of embodiments B1-B12, further comprising:performing a fisheye transformation, a conformal mapping transformation,or other transformation on the common region, wherein generating theenhanced image comprises generating the enhanced image based on thecommon region and the divergent regions subsequent to the performance ofthe foregoing transformation(s).B14. The method of any of embodiments B1-B13, further comprising:determining a defective visual field portion of a visual field of theuser, wherein the visual field of the user comprises visual fieldportions, the defective visual field portion being one of the visualfield portions that fails to satisfy one or more vision criteria, andwherein generating the enhanced image based on the determined defectivevisual field portion such that at least one of the common region or thedivergent regions in the enhanced image do not overlap with thedefective visual field portion of the visual field of the user.B15. The method of any of embodiments B1-B14, further comprising:determining a visual field portion of the user's visual field thatsatisfies (i) one or more vision criteria, (ii) one or more positioncriteria, and (iii) one or more size criteria, and wherein generatingthe enhanced image based on the visual field portion such that at leastone of the common region or the divergent regions in the enhanced imageis within the visual field portion.B16. The method of embodiment B15, wherein the one or more size criteriacomprises a requirement that the visual field portion be a largestvisual field portion of the user's visual field that satisfies the oneor more vision criteria and the one or more position criteria.B17. The method of any of embodiments B15-B16, wherein the one or moreposition criteria comprises a requirement that a center of the visualfield portion correspond to a point within a macular region of an eye ofthe user.B18. The method of any of embodiments B1-B17, wherein one or more of theforegoing operations are performed by a wearable device.B19. The method of embodiment B18, further comprising: causing one ormore display portions of the wearable device to be transparent, whereincausing the enhanced image to be displayed comprises causing an enhancedimage to be displayed on one or more other display portions of thewearable device other than the one or more transparent display portions.B20. The method of embodiment B19, further comprising: causing anadjustment of the one or more transparent display portions and the oneor more other display portions of the wearable device.B21. The method of embodiment B20, further comprising: monitoring one ormore changes related to one or more eyes of the user, wherein causingthe adjustment comprises causing, based on the monitoring, theadjustment of the one or more transparent display portions and the oneor more other display portions of the wearable device.B21. The method of embodiment B20, further comprising: monitoring one ormore changes related to one or more eyes of the user, wherein causingthe adjustment comprises causing, based on the monitoring, theadjustment of the one or more transparent display portions and the oneor more other display portions of the wearable device.B22. The method of any of embodiments B20-B21, wherein causing theadjustment comprises causing an adjustment of one or more positions,shapes, sizes, or transparencies of the one or more transparent displayportions of the wearable device based on the monitoring.B23. The method of any of embodiments B20-B22, wherein the enhancedimage or the adjustment is based on the one or more changes.B24. The method of any of embodiments B18-B23, wherein causing theenhanced image to be displayed comprises causing one or more of thecommon region or the divergent regions to be displayed on the one ormore other display portions of the wearable device such that at leastone of the common region or the divergent regions are not displayed onthe one or more transparent display portions of the wearable device.B25. The method of any of embodiments B18-B24, wherein the wearabledevice comprises first and second cameras, and wherein obtaining theimages comprises obtaining at least one of the images via the firstcamera of the wearable device and obtaining at least another one of theimages via the second camera of the wearable device.B26. The method of any of embodiments B18-B25, wherein the one or moremonitors of the wearable device comprises first and second monitors, andwherein causing the enhanced image to be displayed comprises causing theenhanced image to be displayed via the first and second monitors.B27. The method of any of embodiments B18-B26, wherein the wearabledevice comprises a wearable spectacles device.B28. The method of any of embodiments B1-B27, wherein the enhanced imageor the adjustment is based on feedback related to a set of stimuli(e.g., the feedback indicating whether or how the user senses one ormore stimuli).C1. A method comprising: monitoring one or more changes related to oneor more eyes of a user; causing, based on the monitoring, an adjustmentof one or more transparent display portions or one or more other displayportions of a wearable device; and causing an enhanced image to bedisplayed on the one or more other display portions of the wearabledevice, wherein the enhanced image is based on live image data obtainedvia the wearable device.C2. The method of embodiment C1, wherein causing the adjustmentcomprises causing, based on the monitoring, an adjustment of one or morepositions, shapes, sizes, brightness levels, contrast levels, sharpnesslevels, or saturation levels of the one or more transparent displayportions of the wearable device or the one or more other displayportions of the wearable device.C3. The method of any of embodiments C1-C2, further comprising:determining a defective visual field portion of a visual field of theuser, wherein the visual field of the user comprises visual fieldportions, the defective visual field portion being one of the visualfield portions that fails to satisfy one or more vision criteria, andwherein causing the adjustment comprises causing an adjustment of one ormore positions, shapes, or sizes of the one or more transparent displayportions of the wearable device such that the one or more transparentdisplay portions do not overlap with the defective visual field portion.C4. The method of embodiment C3, further comprising: detecting an object(e.g., in the defective visual field portion or predicted to be in thedefective visual field portion); determining that the object is notsufficiently in any image portion of the enhanced image that correspondsto at least one of the visual field portions satisfying one or morevision criteria; generating a prediction indicating that the object willcome in physical contact with the user; and causing an alert to bedisplayed (e.g., over the enhanced image) based on (i) the prediction ofphysical contact and (ii) the determination that the object is notsufficiently any image portion of the enhanced image that corresponds toat least one of the visual field portions satisfying the one or morevision criteria, wherein the alert indicates an oncoming direction ofthe object.C5. The method of any of embodiments C1-C4, further comprising:providing information related to the one or more eyes to a model, themodel being configured based on the information related to the one ormore eyes; subsequent to the configuring of the model, providing the oneor more monitored changes related to the one or more eyes to the modelto obtain a set of modification parameters, wherein causing theadjustment of the one or more transparent display portions comprisescausing the adjustment of the one or more transparent display portionsbased on one or more modification parameters of the set of modificationparameters.C6. The method of embodiment C5, wherein the information related to theone or more eyes comprises one or more images of the one or more eyes.C7. The method of any of embodiments C5-C6, wherein the informationrelated to the one or more eyes comprises feedback related to a set ofstimuli (e.g., the feedback indicating whether or how the user sensesone or more stimuli).C8. The method of any of embodiments C1-C7, wherein the one or morechanges comprises an eye movement, a change in gaze direction, or apupil size change.C9. The method of any of embodiments C1-C8, wherein the enhanced imageor the adjustment is based on feedback related to a set of stimuli(e.g., the feedback indicating whether or how the user senses one ormore stimuli).C10. The method of any of embodiments C1-C9, wherein the enhanced imageor the adjustment is based on the one or more changes.C11. The method of any of embodiments C1-C10, wherein the adjustment isperformed simultaneously with the display of the enhanced image.C12. The method of any of embodiments C1-C11, wherein one or more of theforegoing operations are performed by the wearable device.C13. The method of any of embodiments C1-C12, wherein the wearabledevice comprises a wearable spectacles device.D1. A method comprising: monitoring one or more eyes of a user (e.g.,during a first monitoring period in which a set of stimuli are displayedto the user); obtaining feedback related to the set of stimuli (e.g.,during the first monitoring period); and generating a set ofmodification profiles associated with the user based on the feedbackrelated to the set of stimuli, each modification profile of the set ofmodification profiles (i) being associated with a set of eye-relatedcharacteristics and (ii) comprising one or more modification parametersto be applied to an image to modify the image for the user wheneye-related characteristics of the user match the associated set ofeye-related characteristics.D2. The method of embodiment D1, wherein the feedback related to the setof stimuli indicates whether or how the user sees one or more stimuli ofthe set of stimuli.D3. The method of any of embodiments D1-D2, wherein the feedback relatedto the set of stimuli comprises one or more characteristics related tothe one or more eyes occurring when the one or more stimuli aredisplayed (e.g., during the first monitoring period).D4. The method of any of embodiments D1-D3, further comprising:monitoring the one or more eyes of the user (e.g., during a secondmonitoring period); obtaining image data representing an environment ofthe user (e.g., during the second monitoring period); obtaining one ormore modification profiles associated with the user based on (i) theimage data or (ii) characteristics related to the one or more eyes(e.g., from the second monitoring period); and causing modified imagedata to be displayed to the user (e.g., during the second monitoringperiod) based on (i) the image data and (ii) the one or moremodification profiles.D5. The method of embodiment D4, wherein the characteristics related tothe one or more eyes comprises gaze direction, pupil size, limbusposition, visual axis, optical axis, or eyelid position or movement.D6. The method of any of embodiments D1-D5, wherein obtaining thefeedback related to the set of stimuli comprises obtaining an eye imagecaptured during the first monitoring period, the eye image being animage of an eye of the user, and wherein generating the set ofmodification profiles comprises generating the set of modificationprofiles based on the eye image.D7. The method of embodiment D6, wherein the eye image is an image of aretina of the eye or an image of a cornea of the eye.D8. The method of any of embodiments D1-D7, wherein obtaining thefeedback related to the set of stimuli comprises obtaining an indicationof a response of the user to the one or more stimuli or an indication ofa lack of response of the user to the one or more stimuli, and whereingenerating the set of modification profiles comprises generating the setof modification profiles based on the indication of the response or theindication of the lack of response.D9. The method of embodiment D8, wherein the response comprises an eyemovement, a gaze direction, or a pupil size change.D10. The method of any of embodiments D1-D9, wherein one or more of theforegoing operations are performed by a wearable device.D11. The method of embodiment D10, wherein the wearable device comprisesa wearable spectacles device.E1. A method comprising: causing a first stimulus to be displayed at afirst interface location on a user interface of a user based on afixation point for a visual test presentation; adjusting, during thevisual test presentation, the fixation point for the visual testpresentation based on eye characteristic information related to theuser, the eye characteristic information indicating one or morecharacteristics related to one or more eyes of the user that occurredduring the visual test presentation; causing a second stimulus to bedisplayed at a second interface location on the user interface based onthe adjusted fixation point for the visual test presentation; obtainingfeedback information indicating feedback related to the first stimulusand feedback related to the second stimulus, the feedback related to thefirst or second stimulus indicating a response of the user or lack ofresponse of the user to the first or second stimulus; and generatingvisual defect information associated with the user based on the feedbackinformation.E2. The method of embodiment of E1, the user interface is configured todisplay a view having a horizontal dimension corresponding to a firstnumber of degrees or a vertical dimension corresponding the first numberof degrees, and wherein the visual defect information is generated suchthat the visual defect information has coverage for greater than thefirst number of degrees with respect to the horizontal dimension for thevisual field of the user or with respect to the vertical dimension forthe visual field of the user.E3. The method of any of embodiments E1-E2, wherein the user interfaceis configured to display a view having a given dimension correspondingto a first number of degrees, and wherein the visual defect informationis generated such that (i) the visual defect information indicates atleast two defects existing at visual field locations of a visual fieldof the user and (ii) the visual field locations are greater than thefirst number of degrees apart with respect to the given dimension forthe visual field of the user.E4. The method of any of embodiments E1-E3, wherein the user interfaceis configured to display a view having a given dimension correspondingto a first number of degrees, wherein the feedback information furtherindicates feedback related to a third stimulus displayed on the userinterface during the visual test presentation, further comprising:determining whether a vision defect exists at visual field locations ofthe visual field of the user based on the feedback information such thatat least two of the visual field locations are apart from one another bymore than the first number of degrees with respect to the givendimension for the visual field; and generating the visual defectinformation based on the determination of whether a vision defect existsat the visual field locations.E5. The method of any of embodiments E1-E4, further comprising:determining the first interface location for the first stimulus based onthe fixation point for the visual test presentation and a first relativelocation associated with the first stimulus; and determining the secondinterface location for the second stimulus based on the adjustedfixation point for the visual test presentation and a second relativelocation associated with the second stimulus, wherein causing firststimulus to be displayed comprises causing, during the visual testpresentation, the first stimulus to be displayed at the first interfacelocation on the user interface based on the determination of the firstinterface location, and wherein causing second stimulus to be displayedcomprises causing, during the visual test presentation, the secondstimulus to be displayed at the second interface location on the userinterface based on the determination of the second interface location.E6. The method of any of embodiments E1-E5, further comprising:selecting, during the visual test presentation, the first interfacelocation for the first stimulus based on the first interface locationbeing farther from the fixation point than one or more other interfacelocations on the user interface, the one or more other interfacelocations corresponding to one or more other visual field locations ofthe test set, wherein causing first stimulus to be displayed comprisescausing, during the visual test presentation, the first stimulus to bedisplayed at the first interface location on the user interface based onthe selection of the first interface location.E7. The method of embodiment E6, further comprising: removing the firstvisual field location from the test set.E8. The method of embodiment E7, wherein removing the first visual fieldlocation comprises removing the first visual field location from thetest set such that the first visual field location is no longeravailable to be selected from the test set during the visual testpresentation.E9. The method of any of embodiments E7-E8, further comprising:selecting, subsequent the removal of the first visual field locationfrom the test set, the second interface location for the second stimulusbased on the second interface location being farther from the adjustedfixation point than the one or more other interface location, whereincausing second stimulus to be displayed comprises causing, during thevisual test presentation, the second stimulus to be displayed at thesecond interface location on the user interface based on the selectionof the second interface location.E10. The method of any of embodiments E6-E9, wherein selecting the firstinterface location comprises selecting the first interface location forthe first stimulus based on the first interface location being at leastas far from the fixation point than all other interface locations on theuser interface that correspond to a visual field location of the testset other than the first visual field position with respect to a givendimension.E11. The method of any of embodiments E6-E10, wherein selecting thesecond interface location comprises selecting the second interfacelocation for the second stimulus based on the second interface locationbeing as least as far from the adjusted fixation point than all otherinterface locations on the user interface that correspond to a visualfield location of the test set other than the second visual fieldposition with respect to a given dimension.E12. The method of any of embodiments E1-E11, further comprising:establishing a lock of the adjusted fixation point such that fixationpoint readjustment is avoided while the lock of the adjusted fixationpoint remains established; causing, while the lock of the adjustedfixation point remains established, one or more stimuli to be displayedon the user interface based on the adjusted fixation point; andreleasing the lock of the adjusted fixation point prior to the displayof the second stimulus.E13. The method of any of embodiments E1-E12, further comprising:causing, while the adjusted fixation point remains the same (e.g., atthe first interface location), multiple stimuli to be displayed on theuser interface and then deemphasized on or removed from the userinterface, wherein at least one stimulus of the multiple stimuli isdisplayed on the user interface subsequent to at least one other stimuliof the multiple stimuli being displayed on the user interface.E14. The method of embodiment E13, wherein the multiple stimuli aredisplayed and then deemphasized or removed while the first stimuluscontinues to be displayed at the first interface location on the userinterface.E15. The method of any of embodiments E13-E14, further comprising:causing the first stimulus to be deemphasized on or removed from theuser interface and then emphasized or redisplayed at the first interfacelocation on the user interface subsequent to at least one stimulus ofthe multiple stimuli being displayed on the user interface.E16. The method of any of embodiments E1-E15, wherein the eyecharacteristic information indicates one or more gaze directions, pupilsize changes, eyelid movements, head movements, or other eye-relatedcharacteristics of the user that occurred during the visual testpresentation.F1. A method comprising: monitoring eye-related characteristics relatedto eyes of a user during visual test presentation via two or more userinterfaces (e.g., on two or more displays) that are provided to therespective eyes, the eyes comprising first and second eyes of the user;causing one or more stimuli to be presented at one or more positions onat least one of the user interfaces; and determining visual defectinformation for the first eye based on one or more eye-relatedcharacteristics (e.g., of the first eye) occurring upon the stimuluspresentation.F2. The method of embodiment F1, wherein determining the visual defectinformation comprises determining a deviation measurement for the firsteye based on one or more eye-related characteristics of the first eyeoccurring upon the stimulus presentation.F3. The method of embodiment F2, wherein deviation measurement indicatesa deviation of the first eye relative to the second eye.F4. The method of any of embodiments F1-F3, wherein causing the stimuluspresentation comprises causing a stimulus to be presented at a firsttime at a position on a first user interface for the first eye such thatthe stimulus presentation occurs while a stimulus is not presented on asecond user interface for the second eye.F5. The method of any of embodiments F1-F4, wherein causing the stimuluspresentation comprises causing a stimulus to be presented at a positionon the first user interface while a stimuli intensity of the second userinterface does not satisfy a stimuli intensity threshold.F6. The method of any of embodiments F4-F5, further comprising: causinga stimulus to be presented at the position on the second user interfaceat a prior time (prior to the first time) while a stimulus is notpresented on the first user interface.F7. The method of any of embodiments F4-F6, further comprising: causinga stimulus to be presented at the first position on the first displayand a stimulus to be presented at the first position on the seconddisplay at a prior time prior to the first time; detecting lack offixation of the first eye on the first position upon the presentation ofa stimulus on the first display at the prior time; and determining thefirst eye of the user to be a deviating eye based on the detection ofthe lack of fixation of the first eye.F8. The method of any of embodiments F4-F7, further comprising: causing,based on the visual defect information (e.g., the deviationmeasurement), a stimulus to be presented at a modified position on thefirst display at a subsequent time subsequent to the first time suchthat the presentation at the subsequent time occurs while a stimulus isnot presented on the second display, the modified position beingdifferent from the first position; and confirming the visual defectinformation (e.g., the deviation measurement) based on one or moreeye-related characteristics of the first eye or the second eye notchanging beyond a change threshold upon the presentation at thesubsequent time.F9. The method of embodiment F8, further comprising: determining, basedon the visual defect information (e.g., the deviation measurement), themodified position as a position at which a stimulus is to be presentedon the first display at the subsequent time.F10. The method of any of embodiments F1-F2, wherein causing thestimulus presentation comprises causing a stimulus to be presented at agiven time at a position on a first user interface for the first eye andat the corresponding position on a second user interface for the secondeye.F11. The method of any of embodiments F1-F10, further comprising:generating a modification profile associated with the user based on thevisual defect information (e.g., the deviation measurement), themodification profile comprising one or more modification parameters tobe applied to modify an image for the user.F12. The method of embodiment F11, further comprising: causing modifiedvideo stream data to be displayed to the user based on (i) video streamdata representing an environment of the user and (ii) the modificationprofile associated with the user.F13. The method of embodiment F12, wherein the modification profilecomprises a translation or rotation parameter to be applied to modify animage for the first eye when the second eye's gaze direction is directedat the first position, wherein causing the modified video stream data tobe displayed comprises: detecting the second eye's gaze direction beingdirected at the first position; using the translation or rotationparameter to modify the video stream data based on the detection of thesecond eye's gaze direction to generate the modified video stream data;and causing the modified video stream data to be displayed to the firsteye of the user.F14. The method of any of embodiments F1-F13, further comprising:generating a first modification profile associated with the user basedon the deviation measurement, the first modification profile comprisingone or more modification parameters to be applied to modify an image forthe first eye in response to the second eye's gaze direction beingdirected at the first position; and generating a second modificationprofile based on a second deviation measurement for the first eye, thesecond modification profile comprising one or more modificationparameters to be applied to modify an image for the first eye inresponse to the second eye's gaze direction being directed at a secondposition different from the first position.F15. The method of any of embodiments F1-F14, wherein determining thevisual defect information comprises determining whether the user hasdouble vision or an extent of the double vision based on a number ortype of stimuli seen by the user.F16. The method of embodiment F15, further comprising: determining thenumber or type of stimuli seen by the user based on a user inputindicating the number or type of stimuli that the user sees.F17. The method of any of embodiments F15-F16, further comprising:determining the number or type of stimuli seen by the user based on oneor more eye-related characteristics occurring upon the stimuluspresentation.F18. The method of any of embodiments F1-F17, wherein determining thevisual defect information comprises determining whether the user hasstereopsis or an extent of the stereopsis based on one or moreeye-related characteristics occurring upon the stimulus presentation.F19. The method of any of embodiments F1-F18, wherein the eye-relatedcharacteristics comprises one or more gaze directions, pupil sizechanges, or other eye-related characteristics.G1. A method comprising: obtaining, via a model, (i) a set of predictedcharacteristics for a set of locations of a user interface and (ii) aset of confidence scores associated with the set of locations;selecting, based on the set of confidence scores, one or more locationsof the set of locations that are to be tested during a visual testpresentation, the one or more locations being selected over one or moreother locations of the set of locations based on the set of confidencescores; and causing, based on one or more predicted characteristicsassociated with the selected locations, one or more stimuli to bepresented at the selected locations during the visual test presentation.G2. The method of embodiment G1, further comprising: obtaining feedbackindicating one or more threshold characteristics under which a user seesthe one or more stimuli at the selected locations; and generating visualdefect information for the user based on the feedback.G3. The method of embodiment G2, further comprising: obtaining, based onthe feedback, (i) a second set of predicted characteristics for a secondset of locations of the user interface and (ii) a second set ofconfidence scores associated with the second set of locations;selecting, based on the second set of confidence scores, one or moresecond locations of the second set of locations that are to be testedduring the visual test presentation; causing, based on one or morepredicted characteristics associated with the selected second locations,one or more second stimuli to be presented at the selected secondlocations during the visual test presentation.G4. The method of embodiment G3, further comprising: obtainingadditional feedback indicating one or more threshold characteristicsunder which the user sees the one or more second stimuli; and generatingthe visual defect information based on the feedback and the additionalfeedback.G5. The method of any of embodiments G3-G4, wherein the second set oflocations comprises the one or more other locations of the set oflocations and one or more additional locations.G6. The method of any of embodiments G1-G5, further comprising:obtaining prior feedback indicating one or more thresholdcharacteristics under which a user sees one or more prior stimulipresented on a user interface; and providing the prior feedback to themodel to obtain the set of predicted characteristics and the set ofconfidence scores.G7. The method of any of embodiments G1-G6, wherein presenting the oneor more stimuli at the selected second locations comprises initiallypresenting a first stimulus at a first location of the selectedlocations based on a first predicted characteristic associated with thefirst location, and wherein the first predicted characteristic comprisesa prediction of a threshold characteristic of a range of characteristicsunder which a user sees the first stimulus or a characteristic adjacentthe threshold characteristic within the range of characteristics.G8. The method of any of embodiments G1-G6, wherein presenting the oneor more stimuli at the selected locations comprises initially presentinga first stimulus at a first location of the selected locations under afirst predicted characteristic associated with the first location, andwherein the first predicted characteristic comprises a prediction of athreshold characteristic of the range of characteristics under which auser sees the first stimulus.G9. The method of any of embodiments G1-G6, wherein presenting the oneor more stimuli at the selected locations comprises initially presentinga first stimulus at a first location of the selected locations under atleast one characteristic that is adjacent, in the range ofcharacteristics, to a first predicted characteristic associated with thefirst location, and wherein the first predicted characteristic comprisesa prediction of a threshold characteristic of the range ofcharacteristics under which a user sees the first stimulus.G10. The method any of embodiments G1-G9, wherein selecting the one ormore locations comprises selecting the one or more locations over theone or more other locations based on one or more confidence scoresassociated with the one or more locations being greater than one or moreconfidence scores associated with the one or more other locations.G11. The method any of embodiments G1-10, wherein the one or morepredicted characteristics associated with the selected locationscomprises a predicted contrast level, a predicted saturation level, or apredicted sharpness level.G12. The method of any of embodiments G1-G11, further comprising:initializing a ground truth input for the model, the ground truth inputcomprising initial feedback related to an initial set of stimulipresented at an initial set of locations of a user interface under arange of characteristics, the initial feedback indicating thresholdcharacteristics of the range of characteristics under which a user seeseach stimulus of the initial set of stimuli; and providing the groundtruth input to the model to obtain the set of predicted characteristicsand the set of confidence scores.H1. A tangible, non-transitory, machine-readable medium storinginstructions that, when executed by a data processing apparatus, causethe data processing apparatus to perform operations comprising those ofany of embodiments A1-A34, B1-B28, C1-C13, D1-D11, E1-E16, F1-F19, orG1-G12.H2. A system comprising: one or more processors; and memory storinginstructions that, when executed by the processors, cause the processorsto effectuate operations comprising those of any of embodiments A1-A34,B1-B28, C1-C13, D1-D11, E1-E16, F1-F19, or G1-G12.

1-20. (canceled)
 21. A system for facilitating vision testing viapredicted-pattern-based selection of a testing location subset, thesystem comprising: a computer system that comprises one or moreprocessors executing computer program instructions that, when executed,cause the computer system to perform operations comprising:initializing, during a visual test presentation comprising multiple testrounds, a ground truth input for a neural network, the ground truthinput comprising initial feedback related to an initial set of stimulipresented at an initial set of locations of a user interface of awearable device under a range of characteristics, the initial feedbackindicating threshold characteristics of the range of characteristicsunder which a user sees each stimulus of the initial set of stimuli;performing the following during each round of the multiple test rounds:providing, during the round of the visual test presentation, the groundtruth input to the neural network to cause the neural network togenerate a pattern of predicted characteristics, the pattern ofpredicted characteristics corresponding to a set of predictedcharacteristics for a set of locations of the user interface;presenting, based on the pattern of predicted characteristics, stimuliat locations of the set of locations during the round such that: (i) afirst stimulus is initially presented at a first location of thelocations based on a first predicted characteristic associated with thefirst location; and (ii) a second stimulus is initially presented atsecond location of the locations based on a second predictedcharacteristic associated with the second location; obtaining feedbackrelated to the stimuli during the round, the feedback related to thestimuli indicating threshold characteristics of the range ofcharacteristics under which the user sees each stimulus of the stimuli;and updating, based on the feedback related to the stimuli, the groundtruth input for a next round of the visual test presentation such thatthe updated ground truth input comprises the initial feedback and thefeedback related to the stimuli; and generating visual defectinformation for the user based on the initial feedback and the feedbackfrom the multiple test rounds of the visual test presentation.
 22. Thesystem of claim 21, wherein the first stimulus is initially presented atthe first location under the first predicted characteristic.
 23. Thesystem of claim 21, wherein the first stimulus is initially presented atthe first location under at least one characteristic that is adjacent,in the range of characteristics, to the first predicted characteristic.24. The system of claim 21, wherein the range of characteristicscomprises contrast levels, saturation levels, or sharpness levels.
 25. Amethod comprising: initializing a ground truth input for a predictionmodel, the ground truth input comprising initial feedback related to aninitial set of stimuli presented at an initial set of locations of auser interface under a range of characteristics, the initial feedbackindicating threshold characteristics of the range of characteristicsunder which a user sees each stimulus of the initial set of stimuli;obtaining, via the prediction model, based on the ground truth input, apattern of predicted characteristics, the pattern of predictedcharacteristics corresponding to a set of predicted characteristics fora set of locations of the user interface; presenting, based on thepattern of predicted characteristics, one or more stimuli at one or morelocations of the set of locations during a portion of a visual testpresentation; obtaining feedback related to the one or more stimuliduring the portion of the visual test presentation, the feedbackindicating one or more threshold characteristics of the range ofcharacteristics under which the user sees each stimulus of the one ormore stimuli; updating, based on the feedback related to the one or morestimuli, the ground truth input for a subsequent portion of the visualtest presentation such that the updated ground truth input comprises thefeedback related to the one or more stimuli; and generating visualdefect information for the user based on the initial feedback, thefeedback from the portion of the visual test presentation, andadditional feedback from the subsequent portion of the visual testpresentation.
 26. The method of claim 25, wherein presenting the one ormore stimuli comprises initially presenting a first stimulus at a firstlocation of the set of locations based on a first predictedcharacteristic associated with the first location, and wherein the firstpredicted characteristic comprises a prediction of a thresholdcharacteristic of the range of characteristics under which the user seesthe first stimulus or a characteristic adjacent the thresholdcharacteristic within the range of characteristics.
 27. The method ofclaim 25, wherein presenting the one or more stimuli comprises initiallypresenting a first stimulus at a first location of the set of locationsunder a first predicted characteristic associated with the firstlocation, and wherein the first predicted characteristic comprises aprediction of a threshold characteristic of the range of characteristicsunder which the user sees the first stimulus.
 28. The method of claim25, wherein presenting the one or more stimuli comprises initiallypresenting a first stimulus at a first location of the set of locationsunder at least one characteristic that is adjacent, in the range ofcharacteristics, to a first predicted characteristic associated with thefirst location, and wherein the first predicted characteristic comprisesa prediction of a threshold characteristic of the range ofcharacteristics under which the user sees the first stimulus.
 29. Themethod of claim 25, further comprising: obtaining, via the predictionmodel, based on the ground truth input, (i) a set of patterns ofpredicted characteristics and (ii) a set of confidence scores associatedwith the set of patterns, wherein each pattern of the set of patternsindicates a predicted characteristic for a same location on the userinterface that is different from a predicted characteristic for the samelocation indicated by another pattern of the set of patterns; andselecting, based on the set of confidence scores, the pattern to be usedfor the portion of the visual test presentation, the pattern beingselected over one or more other patterns of the set of patterns based ona confidence score associated with the pattern being greater than one ormore confidence scores associated with the one or more other patterns,wherein the presentation of the one or more stimuli is based on theselection of the pattern.
 30. The method of claim 29, wherein obtainingthe set of patterns of predicted characteristics and the set ofconfidence scores comprises obtaining the set of patterns of predictedcharacteristics and the set of confidence associated with the set ofpatterns from the prediction model subsequent to providing the groundtruth input to the prediction model.
 31. The method of claim 29, furthercomprising: obtaining, based on the updated ground truth input, (i) asecond pattern of predicted characteristics for a second set oflocations of the user interface and (ii) a second set of confidencescores associated with the second pattern of predicted characteristics;presenting, based on the second pattern of predicted characteristics,one or more second stimuli at one or more second locations during thesubsequent portion of the visual test presentation; and obtaining theadditional feedback during the subsequent portion of the visual testpresentation, the additional feedback indicating one or more thresholdcharacteristics of the range of characteristics under which the usersees each stimulus of the one or more second stimuli.
 32. The method ofclaim 25, wherein the range of characteristics comprises contrastlevels, saturation levels, or sharpness levels.
 33. The method of claim25, wherein the prediction model comprises one or more neural networks.34. A non-transitory computer-readable media storing instructions that,when executed by one or more processor, cause operations comprising:obtaining first feedback indicating one or more thresholdcharacteristics under which a user sees one or more first stimulipresented on a user interface; obtaining, via a prediction model, basedon the first feedback, a pattern of predicted characteristics, thepattern of predicted characteristics corresponding to a set of predictedcharacteristics for a set of locations of the user interface;presenting, based on the pattern of predicted characteristics, one ormore stimuli at one or more locations of the set of locations during avisual test presentation; obtaining second feedback indicating one ormore threshold characteristics under which the user sees the one or morestimuli; and generating visual defect information for the user based onthe first feedback and the second feedback.
 35. The media of claim 34,wherein presenting the one or more stimuli comprises initiallypresenting a first stimulus at a first location of the set of locationsbased on a first predicted characteristic associated with the firstlocation, and wherein the first predicted characteristic comprises aprediction of a threshold characteristic of a range of characteristicsunder which the user sees the first stimulus or a characteristicadjacent the threshold characteristic within the range ofcharacteristics.
 36. The media of claim 34, wherein presenting the oneor more stimuli comprises initially presenting a first stimulus at afirst location of the set of locations under a first predictedcharacteristic associated with the first location, and wherein the firstpredicted characteristic comprises a prediction of a thresholdcharacteristic of a range of characteristics under which the user seesthe first stimulus.
 37. The media of claim 34, wherein presenting theone or more stimuli comprises initially presenting a first stimulus at afirst location of the set of locations under at least one characteristicthat is adjacent, in a range of characteristics, to a first predictedcharacteristic, and wherein the first predicted characteristic comprisesa prediction of a threshold characteristic of the range ofcharacteristics under which the user sees the first stimulus.
 38. Themedia of claim 34, the operations further comprising: obtaining, via theprediction model, based on the ground truth input, (i) a set of patternsof predicted characteristics and (ii) a set of confidence scoresassociated with the set of patterns, wherein each pattern of the set ofpatterns indicates a predicted characteristic for a same location on theuser interface that is different from a predicted characteristic for thesame location indicated by another pattern of the set of patterns; andselecting, based on the set of confidence scores, the pattern to be usedfor the visual test presentation, the pattern being selected over one ormore other patterns of the set of patterns based on a confidence scoreassociated with the pattern being greater than one or more confidencescores associated with the one or more other patterns, wherein thepresentation of the one or more stimuli is based on the selection of thepattern.
 39. The media of claim 34, the operations further comprising:obtaining, based on the updated ground truth input, (i) a second patternof predicted characteristics for a second set of locations of the userinterface and (ii) a second set of confidence scores associated with thesecond pattern of predicted characteristics; presenting, based on thesecond pattern of predicted characteristics, one or more second stimuliat one or more second locations of the second set of locations duringthe visual test presentation; and obtaining the additional feedbackduring the visual test presentation, the additional feedback indicatingone or more threshold characteristics of the range of characteristicsunder which the user sees each stimulus of the one or more secondstimuli.
 40. The media of claim 34, wherein the range of characteristicscomprises contrast levels, saturation levels, or sharpness levels.